Question

I’m trying to scrape a website and do I need to use Javascript. I am using Python 2.7 and Spynner. This is the script:

   <a href="javascript://" onclick="TranslationPopUpTestimonialQuery('Language_ALL', '',       '5', '10', 'date-of-visit')">6</a>

This is my code in Python:

   import spynner
   browser = spynner.Browser()
   browser.load("http://www.zoover.it/italia/lazio/roma/galileo/hotel")
   page1 = browser._get_html()
   browser.runjs("TranslationPopUpTestimonialQuery('Language_ALL', '', '8', '10', 'date-of-visit')")
   browser.wait_load()
   page2 = browser._get_html()

Something does not work as get_html does not give me the desired html page…

This the first url: http://www.zoover.it/italia/lazio/roma/galileo/hotel. I load this page with browser.load() and i get the HTML. After i try to go to page 2 simulating a click with browser.runjs(). I call the javascript on the onClick. Then If i call browser._get_html() another time the HTML is ever the same.
Can anyone help me? thanks

Answer 1

It the html you want is in a popup, you ll have to switch spynner frames.

What spynner gives you is always the active frame html, popups open in another frames.

Spynner is an interface to qtwebwekit and qtwebkit use that : http://doc.qt.digia.com/4.7-snapshot/qwebframe.html

You can see that there is “childframes”, that you can introspect with self.webframe.