Question

I’m trying to grab some data using Selenium and xpaths.

The following xpath works fine:

print sel.get_attribute("xpath=(//*[@id='course_list']/*[@class='series'])[4]//*[@class='series_links']//a/@href")

and returns 4 matching URL’s. So far so good.

The issue is that I want to write an xpath to target each URL individually.

Using the Firefox xpath checker plugin, I have managed to confirm that the following code does exactly what I need:

((//*[@id='course_list']/*[@class='series'])[4]//*[@class='series_links']//a/@href)[1]

But despite working in the Firefox xpath checker, I can’t seem to get this to work in Selenium.

Whenever I try to execute:

print sel.get_attribute("xpath=((//*[@id='course_list']/*[@class='series'])[4]//*[@class='series_links']//a/@href)[1]")

I get the following error:

Exception: ERROR: Invalid xpath [2]: ((//*[@id='course_list']/*[@class='series'])[4]//*[@class='series_links']//a

Not sure what’s going on here. Am I making a simple mistake, or do Selenium xpath’s not support nested brackets like the FF xpath checker does?

Any thoughts would be most appreciated, as I’ve been working on this for hours and can’t seem to make it work 🙁

Answer 1

This is, again, not an answer to your question. But, I never use xpaths like this. If the webpage writer was smart enough to use classes, he’s also smart enough to be able to change the structure of the webpage and keep those classes.

from selenium import webdriver
driver = webdriver.Chrome() 

series = driver.find_element_by_class_name("series")
series_links = [i.get_attribute('href') for i in series.find_elements_by_class_name("series_links")]

driver.quit() # call this when you're done using the webdriver.