Question

I am using Selenium 2.0, Firefox 11.0, and Java to process a table. I have a table element composed of td cells, some which contain text included in a span element, others which contain input elements which have text in their value attributes. My goal is to get the text of every cell so I can output the table contents and compare them against expected values. I thought I would just do something like this:

Locate the table WebElement by id
List<WebElement> cells = tableElem.findElements(By.xpath(".//td"));

Then I would loop through all the cells and run findElements with the xpath “.//input” and if the list was empty I would run getText on the webElement, and if the list wasn’t empty I would run getAttribute on the input element.

But to my surprise, this took several minutes to run on firefox (I’m afraid to try it on IE, which is where its supposed to be tested). When I debug it is obvious that the bottleneck is the .//input search from the td which is killing me. It is upwards of ten seconds, and so even with just a few cells my tests are taking forever. I’ve tried all sorts of minor variations to the xpath, tried going to css selectors, and continue to get the same results.

I want some advice about how to either tackle this problem differently or how to optimize my current method. I was hoping this would only take a couple of seconds.

I’ve included some sample code that should illustrate the slowdown I’m experiencing. This is not the website I’m screen scraping, but the slowness is the same:

    webDriver.navigate().to("https://accounts.google.com/NewAccount");
    List<WebElement> TDxpath = webDriver.findElements(By.xpath("//td"));
    List<WebElement> TDcss = webDriver.findElements(By.cssSelector("td"));
    for (WebElement td : TDcss) {
        List<WebElement> q = td.findElements(By.cssSelector("input"));
    }
    for (WebElement td : TDxpath) {
        List<WebElement> r = td.findElements(By.xpath(".//input"));
    }

Answer 1

Do you really need a browser? You could try HtmlUnitDriver, that will be blazingly fast!

Or you could do it as a JS, that also only takes a fraction of time and you can get Lists from the script:

(JavascriptExecutor)driver.executeScript(
    "var tds = document.getElementsByTagName('td');"
    "for (var i = 0; i < tds.length; i++) {" +
    "   var inputs = tds[i].getElementsByTagName('input');" +
    "}"
    );