Question

i need some help trying to figure out how to use variables in xpath attributes when using the scrapy framework

eg hxs.select(‘/tr[@name=”tag_1″]/td/text()’).extract()

but there are 100 tags so tag_1, tag_2, tag_3 etc
im trying to loop through it but scrapy is erroring on me
im using @name=”tag_” + x
where x is the loop variable but its not working
any ideas?

Answer 1

I think that may be just a problem with badly placed quotes, try something like:

for x in range(100):
    str_selector = '/tr[@name="tag_{0}"]/td/text()'.format(x)
    hxs.select(str_selector)