Question

I keep getting an error: invaled syntax for

1.add_xpath('tagLine', '//p[@class="tagline"]/text()')

and I cannot seem to figure out why it is giving me that error, since as far as i can tell it is the same syntax as all of the other 1.add_xpath() methods. my other question is how do I request other pages. basically I am going through one big page and having it go through each link on the page, then once it is done with the page I want it to go to the next (button) for the next large page, but I don’t know how to do that.

   def parse(self, response):
    hxs = HtmlXPathSelector(response)
    for url in hxs.select('//a[@class="title"]/@href').extract():
        yield Request(url, callback=self.description_page)
    for url_2 in hxs.select('//a[@class="POINTER"]/@href').extract():
        yield Request(url_2, callback=self.description_page)

def description_page(self, response):
    l = XPathItemLoader(item=TvspiderItem(), response=response)
    l.add_xpath('title', '//div[@class="m show_head"]/h1/text()')
    1.add_xpath('tagLine', '//p[@class="tagline"]/text()')
    1.add_xpath('description', '//div[@class="description"]/span')
    1.add_xpath('rating', '//div[@class="score"]/text()')
    1.add_xpath('imageSrc', '//div[@class="image_bg"]/img/@src')
    return l.load_item()

any help on this would be greatly appreciated. I am still a bit of a noob when it comes to python and scrapy.

Answer 1

def description_page(self, response):
    l = XPathItemLoader(item=TvspiderItem(), response=response)
    l.add_xpath('title', '//div[@class="m show_head"]/h1/text()')
    1.add_xpath('tagLine', '//p[@class="tagline"]/text()')
    1.add_xpath('description', '//div[@class="description"]/span')
    1.add_xpath('rating', '//div[@class="score"]/text()')
    1.add_xpath('imageSrc', '//div[@class="image_bg"]/img/@src')
    return l.load_item()

You have digit 1 instead of variable name l.