Question

My PHP project works by grabbing data from a WCF REStful web service with curl.

However, for some reason the curl_exec function always takes around 100ms to return, even though the WCF web service is on the same machine, and the WCF service function being called only takes 3ms to execute.

I’ve profiled the WCF service and the entire WCF stack accepts a HTTP request and returns the resultant XML (a tiny XML structure with 5 elements) in under 10ms.

Below is my cURL set-up code:

    $curl = curl_init($this->uri);

    curl_setopt($curl, CURLOPT_HEADER        , true ); // return response HTTP headers in output
    curl_setopt($curl, CURLOPT_RETURNTRANSFER, true ); // return result instead of echoing
    curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); // stop cURL from verifying the peer's certificate. This is set because by default (and right now) cURL does not trust /any/ certificate.
    curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true ); // follow redirects, Location: headers (but does it really?)
    curl_setopt($curl, CURLOPT_MAXREDIRS     , 10   ); // but dont redirect more than 10 times

    curl_setopt($curl, CURLOPT_CUSTOMREQUEST , $this->method );

    if( isset( $this->username  ) )
        curl_setopt($curl, CURLOPT_USERPWD  , $this->username . ':' . $this->password );

    if( isset( $this->userAgent ) )
        curl_setopt($curl, CURLOPT_USERAGENT, $this->userAgent);

    if( isset( $this->content ) ) {
        curl_setopt($curl, CURLOPT_POSTFIELDS, $this->content );

        array_push( $this->headers, 'Content-Length: '. $this->contentLength );

        if( isset( $this->contentType ) )
            array_push( $this->headers, 'Content-Type: ' . $this->contentType );
    }

    if( isset( $this->contentFile ) ) {
        curl_setopt($curl, CURLOPT_INFILE    , $this->contentFile   );
        curl_setopt($curl, CURLOPT_INFILESIZE, $this->contentLength );
        array_push( $this->headers, 'Content-Length: '. $this->contentLength );
        array_push( $this->headers, 'Content-Type: '  . $this->contentType );
    }

    if( count( $this->headers ) > 0 )
        curl_setopt($curl, CURLOPT_HTTPHEADER, $this->headers );

The call to curl_exec happens immediately after this block of code, and it’s the curl_exec call that I measure at 100ms consistently.

It’s important I get the load times of this PHP site down, and I can’t understand why this happens.

UPDATE:

cURL’s curl_exec always seems to take 100ms regardless of what is being requested, although very interestingly when I make a second curl_exec request (in the same PHP page execution) it completes a lot quicker – around 20ms.

The mystery deepens.

UPDATE 2:

I realised that curl_getinfo() returns an associative-array containing timing information. When I dump the info from the first cURL request I see the following:

'total_time' => 0.125,
'namelookup_time' => 0.125,
'connect_time' => 0.125,
'pretransfer_time' => 0.125,
'starttransfer_time' => 0.125,

Whereas all subsequent requests are like this (or better):

'total_time' => 0.016,
'namelookup_time' => 0,
'connect_time' => 0,
'pretransfer_time' => 0,
'starttransfer_time' => 0.016,

It seems the delay is in namelookup, but I don’t know why DNS lookup takes so long given that Windows (the server OS) already has the DNS information cached and other applications (desktop, ASP.NET, etc) can resolve the name in under 1ms, but cURL does it in 125ms.

What’s so special about cURL that it can’t use the OS’s DNS cache?

Answer 1

I think it’s a bug in the version of cURL included in my PHP installation. After I updated cURL (and PHP) to a more recent version the problem disappeared and curl_exec executes in under 20ms now.