Question

I would like to extract the URL of this type (link text is a number with any number of digits and href is a random text) using an XPath in Scrapy.

• <a href="http://www.example.com/link_to_some_page.html>3</a>
• <a href="http://www.example.com/another_link-abcd.html>45</a>

I could think of something like

HtmlXPathSelector(response).select('//a[matches(text(),"\d+")]/@href')

However it appears that XPath 2.0 isn’t supported and I can’t use regex.

The best single line solution I could search was from this question: xpath expression for regex-like matching? – Is there a better way in scrapy to achieve this?

Answer 1

.select('//a[. != "" and translate(., "0123456789", "") = ""]/@href')