want to crawl http://47.99.86.238/portal/li. the data of this website, use scrapy, to set everything else, only one value to get is None, please take a look at it.
I can match the value using the plug-in myself, but the result printed by storyMale in the program is None.
related codes
/ / Please paste the code text below (do not replace the code with pictures)
-sharp
name = "sister_story_spider"
-sharp
allowed_domains = ["http://47.99.86.238"]
-sharp
start_urls = ["http://47.99.86.238/portal/list/index/id/11.html"]
def parse(self, response):
-sharp
data_list = response.xpath("//div[@id="List"]")
for data_item in data_list:
-sharp
sisterStoryDataItem = SisterStoryDataItem()
-sharp
sisterStoryDataItem["storyFeMale"] = data_item.xpath(".//span//span//text()").extract_first()
print("storyFeMale is: ", sisterStoryDataItem["storyFeMale"])
sisterStoryDataItem["storyMale"] = data_item.xpath("normalize-space(.//div[@class="H-flex-item H-margin-vertical-both-10 H-margin-10"]//div[@class="wz1wz H-theme-font-color-333 H-font-size-14 H-text-horizontal-left H-text-show-row-2"]//text())").extract_first()
print("storyMale is: ".format(sisterStoryDataItem["storyMale"]))
sisterStoryDataItem["storySort"] = data_item.xpath(".//span[@class="H-display-block H-flex-item H-text-align-right H-theme-font-color-999 H-font-size-12"]//text()").extract_first()
print("storySort is: ", sisterStoryDataItem["storySort"])
what result do you expect? What is the error message actually seen?
The expected result of can be matched to a specific value, but it cannot be matched at the moment.
error log is as follows: