我编写了一个 scrapy 程序来从站点抓取数据,如果我抓取到 json 文件或 csv 文件,该程序会成功抓取,当我尝试抓取到 postgres 数据库时会出现问题,下图显示了我收到的错误,我该如何修复错误:
def process_item(self, item, spider):
"""Save deals in the database.
This method is called for every item pipeline component.
"""
self.cur.execute("insert into Deals (Name,Deal_Url,Image_Url,Old_Price,Special_Price,Final_Price) values(%s,%s,%s,%s,%s,%s)",(item['Name'],item['Product_URL'],item['Image_URL'],item['Old_Price'],item['Special_Price'],item['Final_Price']))
self.connection.commit()
return item
当事务中的操作失败时,除了修复恶意操作(可能尝试访问中的任何值)之外,您还需要回滚item
不存在)您可以将插入语句包装在 try/ except 块中:
def process_item(self, item, spider):
"""Save deals in the database.
This method is called for every item pipeline component.
"""
try:
self.cur.execute("insert into Deals (Name,Deal_Url,Image_Url,Old_Price,Special_Price,Final_Price) values(%s,%s,%s,%s,%s,%s)",(item['Name'],item['Product_URL'],item['Image_URL'],item['Old_Price'],item['Special_Price'],item['Final_Price']))
self.connection.commit()
except:
self.connection.rollback()
return item
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)