<a href="testtese" target="_blank" data-bgimage="testtese"></a>
the a tag acquired by the crawler contains href, target, data-bgimage and other attributes, which can be obtained with this.attr.href and this.attr.target, but this.attr.data-bgimage will report a global name "bgimage" is not defined error and solve
.use this.attr ["data-bgimage"]
index page, can be displayed after the first run, but an error will be reported as soon as you run detail page ...
use the send_message and on_message methods to handle situations where multiple task results are returned from a single page, and prepare to override the on_result method for further processing. However, the msg returned by the on_message method is not ...
problem description when there are many pyspider projects, it is always stuck there and cannot run tasks automatically the environmental background of the problems and what methods you have tried it is not possible to add more than one processor f...
config file { "taskdb ": "mysql+taskdb: pyspider:root@47.94.212.235:3306 taskdb ", "projectdb ": "mysql+projectdb: pyspider:root@47.94.212.235:3306 projectdb ", "resultdb ": "mysql+resultdb: pyspider:root@47.94.212.235:3306 resultdb "}1:[W 18...
problem description in the process of using the pyspider frame, the rendered web page will never be displayed properly in the preview area on the left. in the previous Chrome67, the preview page was a little larger, so you can t see it at all when y...
Exception: HTTP 599: Could not resolve host: www.tripadvisor.cn? appears when pyspider is running as shown in the picture, it was good the other day. I can t do it today. I hope you can give me some advice. Thank you very much! url: https: www.tri...
this is the case with multiple tasks created. After clicking run, it will always be stuck in the active state and will not grab the data. It can be run before, but it can t be restarted now ...
it used to be fine, but now it doesn t work. I don t know what the reason is, and Baidu didn t find out why. I asked the boss for help, thank you D:python.ptc > D:python.ptc > pyspider all dazzle anacondalibsitelypackagespyspiderlibsutils.pyride1...
[root@localhost python]-sharp pyspider all [W 180629 07:08:26 run:413] phantomjs not found, continue running without it. [I 180629 07:08:29 result_worker:49] result_worker starting... [I 180629 07:08:31 processor:211] processor starting... [I 180629 07:08...
I used the selector to select the node, but the run did not respond. When I opened the developer option, I found that the li node was not unique, there was another one in front of it. Don t know how to change ...
problem description `C:pythonPython37Scripts > pip3 install pyspider Requirement already satisfied: pyspider in c:pythonpython37libsite-packages (0.3.10) Requirement already satisfied: Flask > = 0.10 in c:pythonpython37libsite-packages (from pyspid...
it is said that on_message can, but I still can t test it. Is there any way to achieve it? def detail_page(self, response): results = json.loads(response.text) for result in results: date = result[ date ] number = response.ur...