Question

‘m trying to crawl some rows from CSV file using CSVFeedSpider
The structure of the file is the next:
id | category | price
I need to crawl the rows which only have a spefic category “paid”
I do the next:

class Outillage_spider(CSVFeedSpider):
name = 'domain.com'
allowed_domains = ['domain.com', 'www.domain.com']
start_urls = ('http://www.domain.com/file.csv',)

delimiter = ';'
headers = ['name', 'category', 'price']

def parse_row(self, response, row):
    categories = ['Bosch','Dolmar','Fein','Hitachi','Karcher','Leman','Makita','SDMO','Ski']
if row['category'] in categories:
        res = {}
        res['name'] = row['name']
        res['price'] = row['price']
        return load_product(res, response)
    else:
  return None

And the next I got:

      File "/home/rolikoff/web/scrapy_projects/local/lib/python2.7/site-packages/Scrapy-0.14.1-py2.7.egg/scrapy/contrib/spiders/feed.py", line 129, in parse_rows
    raise TypeError('You cannot return an "%s" object from a spider' % type(ret).__name__)
exceptions.TypeError: You cannot return an "NoneType" object from a spider

I think it happens when parse_row() returns None. But I’m not sure how to change the fucthion. Do you have any ideas?

Thanks
Dmitry

Answer 1

Try to return empty list or tuple instead None

else:
    return []

And make sure, that load_product returns list, tuple, Item or Request