Question

I’m trying to use Selenium (in Python) to extract some information from a website. I’ve been selecting elements with XPaths but am having trouble using the following-sibling selector. The HTML is as follows:

<span class="metadata">
    <strong>Photographer's Name: </strong>
    Ansel Adams
</span>

I can select “Photographer’s Name” with

In [172]: metaData = driver.find_element_by_class_name('metadata')

In [173]: metaData.find_element_by_xpath('strong').text
Out[173]: u"Photographer's Name:"

I’m trying to select the section of text after the tag (‘Ansel Adams’ in the example). I assumed I could use the following-sibling selector but I receive the following error:

In [174]: metaData.find_element_by_xpath('strong/following-sibling::text()')
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (328, 0))
... [NOTE: Omitted the traceback for brevity] ...
InvalidSelectiorException: Message: u'The given selector strong/following-sibling::text() is either invalid or does not result in a WebElement. The following error occurred:\n[InvalidSelectorError] The result of the xpath expression "strong/following-sibling::text()" is: [object Text]. It should be an element.' 

Any ideas as to why this isn’t working?

Answer 1

@RossPatterson is correct. The trouble is that the text ‘Ansel Adams’ is not a WebElement, so you cannot use find_element or find_elements. If you change your HTML to

<span class="metadata">
    <strong>Photographer's Name: </strong>
    <strong>Ansel Adams</strong>
</span>

then find_element_by_xpath('strong/following-sibling::*[1]').text returns ‘Ansel Adams’.