Question

I’m writing code to navigate a website (as practice before testing our website) with Selenium. I’m attempting to iterate over values in 3 separate drop-down menus and retrieve an end-page with product data.

So far, I’m able to print data from the first first menu and associated data from the second, but only for the first element in the list created by the first menu…That is probably hard to follow, so consider this example:

DropdownA contains values:

A
B
C
D

When ‘A’ is selected, DropdownB is populated with values:

1
2
3

My goal is to cover the entire tree and return values:

A – 1,2,3

B – 2,3,4

C – 3,4,5

etc.

My script bombs after returning:

A – 1,2,3

Here is an example of my code:

from pyvirtualdisplay import Display
from selenium import webdriver
import time

display = Display(visible=0, size=(1024, 768))
display.start()
driver = webdriver.Firefox()
url = 'http://www.website.com'

driver.get(url)
driver.find_element_by_id('ctl00_ContentPlaceHolder1_pc_selector_selMake').click()
make_list = driver.find_elements_by_css_selector('#ctl00_ContentPlaceHolder1_pc_selector_selMake option')
for raw_make in make_list:
    if 'Select' in raw_make.text:
        continue
    make = raw_make.text
    print make
    raw_make.click()
    time.sleep(1)

    model_list = driver.find_elements_by_css_selector('#ctl00_ContentPlaceHolder1_pc_selector_selModel option')
    for raw_model in model_list:
        if 'Select' in raw_model.text:
            continue
        model = raw_model.text

        print ' ', model

driver.close()
display.stop()

The exact error is: selenium.common.exceptions.StaleElementReferenceException: Message: u'Element is no longer attached to the DOM'

I’m very new to Selenium, so if this is a silly question, forgive me. I’ve spent more time than I care to admit trying anything and everything that comes to mind, or that I’ve read online and nothing has helped (partially because most of the examples I’ve encountered are written in Java, and I’m not a Java guy…).

Any help, suggestions, solutions, resources, etc. is greatly appreciated.

Thanks in advance!

Answer 1

Ultimately, it sounds like the DOM Elements you’re iterating over no longer exist. How could this happen, you may ask? I believe the issue arises in the following: You’re grabbing all tags within a drop-down by calling:

driver.find_elements_by_css_selector('#ctl00_ContentPlaceHolder1_pc_selector_selMake option')

You’re then iterating over each tag and clicking on each. My best guess would be the drop-down may be re-generated every time it is clicked on, therefore invalidating the objects you’re iterating over. Hence the successful first iteration, but every iteration thereafter rendering unsuccessful.

Hope this helps!