Question

I tried doing something like this but it doesn’t work!

$ch = curl_init(); 
curl_setopt($ch, CURLOPT_URL, "http://google.com/"); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('GET /search?q=kk HTTP/1.1
Host: www.google.de
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Proxy-Connection: Close
Cookie: PREF=ID=2bb051bfbf00e95b:U=c0bb6046a0ce0334:
Cache-Control: max-age=0
Connection: Close
'));
$response = curl_exec($ch); 
curl_close($ch);   



    echo $response;

Also, Is it possible to make the entire request with just headers without setting the URL? I mean without this?

curl_setopt($ch, CURLOPT_URL, "http://google.com/"); 

Thanks!

Answer 1

I got it to work.

1) Change header Host: www.google.de to Host: www.google.com

Motivation: the host specified in the Host header should match the URL’s host exactly.

2) Use “www.google.com” instead of “google.com”

Motivation: search requests made to google.com will not retrieve search results. You will be told to go to http://www.google.com.

3) Set the full URL into CURLOPT_URL, not just the hostname. E.g. change the CURLOPT_URL to curl_setopt($ch, CURLOPT_URL, "http://www.google.com/search?q=kk");

Motivation: correct usage of cURL API.

4) Remove GET /search?q=kk HTTP/1.1 from CURLOPT_HTTPHEADER — it’s misplaced.

Motivation: correct usage of cURL API.

5) The response will be gzip or deflate compressed. To stop this, remove the Accept-Encoding: gzip, deflate request header.

Motivation: if you tell Google you’re capable of receiving a compressed response, they will send you one. Decompressing an HTTP response is an extra step that you may not want to undertake. It may be easier to deal with the response if it is in an uncompressed text form.