problem description
how to choose different item processing according to different pipeline
the environmental background of the problems and what methods you have tried
there are multiple crawler items in a scrapy, and each crawler project has a different item,. The result of searching on the Internet is to judge the type of the received item and then execute different code. I hope to call the corresponding pipeline function after the judgment. For example, item a will be handed over to pipeline a to deal with.
related codes
/ / Please paste the code text below (do not replace the code with pictures)
from items import AspiderItem, BspiderItem, CspiderItem
class myspiderPipeline(object):
def __init__(self):
pass
def process_item(self, item, spider):
if isinstance(item, AspiderItem):
pass
elif isinstance(item, BspiderItem):
return item
elif isinstance(item, CspiderItem):
print item
return item
class AspiderPipeline(myspiderPipeline):
def __init__(self):
self.file = open("myadata.json", "wb")
def process_item(self, item, spider):
content = json.dumps(dict(item), ensure_ascii=False) + "\n"
self.file.write(content)
return item
def close_spider(self, spider):
self.file.close()
class BspiderPipeline(myspiderPipeline):
passwhat result do you expect? What is the error message actually seen?
I want to know how to call the corresponding pipeline after item type determination. Instantiate the corresponding class and then call the process_item method? If so, will methods such as close_spider of this class be executed automatically?