Question

I want to scrape some google page with file get contents function:

     $encoded='http://www.google.co.il/#hl=en&biw=1440&bih=799&sclient=psy-ab&q=site:'.urlencode("http://stackoverflow.com/");

    echo  file_get_contents($encoded);

When I do this:

echo $encoded;

I get this:

http://www.google.co.il/#hl=en&biw=1440&bih=799&sclient=psy-ab&q=site:http%3A%2F%2Fstackoverflow.com%2F

When I put it in the url.. I get the page that I want

But when I use the function , I simply get the main google page..why does this happen?

Answer 1

It’s because you url uses googles new client side side functionality. Everything after the # is for the client, and the server will serve you the content under http://www.google.co.il/.

update

If you want to use google search results in your program, try using one of their API for custom search. There have been a search webservice for ajax clients under the this url:

http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=your+term+goes+here

It returns results in JSON, the first result is under responseData->results array:

$search_results = json_decode(file_get_contents('http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=your+term+goes+here'), true);
var_export($search_results['responseData']['results'][0]);

But i think this have been deprecated since than (still responds however).