Question

I’m trying to parse a HTML page where the majority of the content is contained in javascript. When I use the Chrome development tools I can see that the div class I’m trying to grab the content from is called div class=doodle-image. However when I either view the page as a source or try to grab it with php:

<?php 
include_once('simple_html_dom.php');
$html = new simple_html_dom();
$html->load_file('http://www.google.com/doodles/finder/2012/All%20doodles');
$doodles = $html->find('.doodle-image');
echo $html;
?>

It returns the frame of the page but contains none of the divs or content. How can I grab the full content of the page?

Answer 1

That’s because the element is empty when your PHP client fetches it, Google is loading in a JSON-object with JavaScript to populate the list of doodles. It does a Ajax-request to this page, and probably you can too.