How does pyspider judge the end of a task?

I now set the crawl to be performed automatically every 30 minutes
clipboard.png

because the data has to be processed before it can be saved to the database, I need to process it after one round of the task.
before I set automatic execution, I used "on_finished" to determine whether the task was completed, but now I don"t seem to call the "on_finished" method after setting automatic execution. Ask the great god to tell me what to do now.

Mar.14,2021

my solution is to write the for loop as detail_page recursion and use save to pass data

save = {
    'current': 3,
    'all': 20,
    'allUrls': [url1, ... url20],
    'results': [result1, result2],
}

if current = = all , perform database operations

MySQL Query : SELECT * FROM `codeshelper`.`v9_news` WHERE status=99 AND catid='6' ORDER BY rand() LIMIT 5
MySQL Error : Disk full (/tmp/#sql-temptable-64f5-1befd37-3a067.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
MySQL Errno : 1021
Message : Disk full (/tmp/#sql-temptable-64f5-1befd37-3a067.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
Need Help?