Ask about the choice of reptiles.

now there is a need to crawl an article from a website, including all the js,css.html files, and then save it to become your own article, which is loaded asynchronously through ajax. So I would like to ask, this kind of demand, which way to achieve better, scrapy splash and puppeteer seem to be similar in principle. In addition to the above two, there is no other framework for my current needs, the language is selected in node and ptyhon for advice.

Html5 node.js python
Apr.19,2022

selenium is good, although inefficient

articles are obtained through ajax , why don't you just use this interface?

finally, I chose puppeteer

. I think that the retro combination of scrapy and bs4 will not fail to apply

dynamic web pages loaded through ajax. It is recommended to use selenium

.

Previous: Is there a conflict in the submit form validation rules when antd Tab is switched?

Next: About Community Profiles on github

css mysql arrays josn react html typescript webpack npm sass R objective-c .net sql-server jquery python-3.x angularjs django angular excel regex iphone ajax linux xml pandas vba spring database wordpress string wpf xcode windows bash postgresql oracle multithreading eclipse list firebase algorithm macos forms image scala visual-studio azure bootstrap spring-boot react-native python-2.7 docker performance function winforms matlab powershell apache dataframe api sqlite numpy rest shell selenium flutter dart maven loops qt swing android-studio csv express file class tensorflow sorting codeigniter perl
MySQL Query : SELECT * FROM `codeshelper`.`v9_news` WHERE status=99 AND catid='6' ORDER BY rand() LIMIT 5
MySQL Error : Disk full (/tmp/#sql-temptable-64f5-1b40c99-2c54d.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
MySQL Errno : 1021
Message : Disk full (/tmp/#sql-temptable-64f5-1b40c99-2c54d.MAI); waiting for someone to free some space... (errno: 28 "No space left on device")
Need Help?