problem description
this is my first contact with the Scrapy framework. When crawling the data in the JSON file in Scrapy, using print detection found to be empty, do not know what is going on, please give advice!
the environmental background of the problems and what methods you have tried
The environment forPython is Python 3.7. Try to say response.text () online is invalid.
related codes
/ / Please paste the code text below (do not replace the code with pictures)
import scrapy
import json
from scrapy.http import Request
from urllib import parse
from MovieSpider.MovieSpider.items import MoviespiderItem
class MovieSpider (scrapy.Spider):
name = "MovieSpider"
allowed_domains = ["movie.douban.com"]
start_urls = ["https://movie.douban.com/j/search_subjects?type=movie&tag=%E5%86%B7%E9%97%A8%E4%BD%B3%E7%89%87&sort=rank&page_limit=20&page_start=0"]
def parse(self, response):
list = json.loads(response.text())
print(list)
what result do you expect? What is the error message actually seen?
No error is reported, but the output is empty. Please give me your advice.