Does pyspider support mongodb clusters as taskdb?

problem description

capture answers similar to Zhihu because there are so many answers from Zhihu, response.save is used to save the results of crawling ahead

because Zhihu site cannot be crawled too fast, the task may not be completed in time

so taskdb needs to access a lot of data. The taskdb of that project is close to 250GB

.

what result do you expect? What is the error message actually seen?

whether pyspider supports mongodb cluster as taskdb
and then how to configure it.

Mar.31,2021

is configured in the document: ider.org/en/latest/Deployment/" rel=" nofollow noreferrer "> http://docs.pyspider.org/en/l.

.
MySQL Query : SELECT * FROM `codeshelper`.`v9_news` WHERE status=99 AND catid='6' ORDER BY rand() LIMIT 5
MySQL Error : Disk full (/tmp/#sql-temptable-64f5-1bef8a1-53320.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
MySQL Errno : 1021
Message : Disk full (/tmp/#sql-temptable-64f5-1bef8a1-53320.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
Need Help?