Question

I’m using cURL in my PHP app to connect to a RESTful API. However, I just recently discovered that I’m not parallelizing my cURL connections and so performing several successive connection results in extreme latency for the end-user.

I’ve not used curl_multi before and I’m kind of at a loss after reading through the documentation. How can I best refactor the following code to take advantage of curl_multi‘s parallelization?

EDIT: I forgot to mention that I open-sourced the API that’s being used here. These are my own Directed Edge PHP bindings. So if you’d like, you can also have your help here merged into the code on GitHub and you’ll be listed as a contributor.

Here’s an example of what I’m doing in the client code:

 // Get 100 goal recommendations from Directed Edge
  $de = new DirectedEdgeRest();
  $item = "user".$uid;
  $limit = 100;
  $tags = "goal";
  $recommendedGoals = $de->getRecommended($item, $tags, $limit);

  // Get 100 interest recommendations from Directed Edge
  $de = new DirectedEdgeRest();
  $item = "user".$uid;
  $limit = 100;
  $tags = "interest";
  $recommendedInterests = $de->getRecommended($item, $tags, $limit);

And here are the relevant functions from DirectedEdgeRest()

  /**
   * Returns array of recommended result IDs for an item
   * @param string $item Item, e.g. "Miles%20Davis"
   * @param string $tags Tags as comma delimited string, e.g. "product,page"
   * @param int $limit Limit for max results
   *
   * @return array Recommended result IDs
   */
  public function getRecommended($item, $tags, $limit)
  {
    // Connect to Directed Edge and parse the returned XML
    $targeturl = self::buildURL($item, 'recommended', $tags, $limit, 'true');
    $response = self::getCurlResponse($targeturl);
    $xml = self::parseXML($response);

    // Iterate through the XML and place IDs into an array
    foreach($xml->item->recommended as $recommended) {
      $recommendedResults[] = filter_var($recommended, FILTER_SANITIZE_NUMBER_INT);
    }

    return $recommendedResults;
  }

  /**
   * Builds URL for cURL
   * @param string $item Item, e.g. "Miles%20Davis"
   * @param string $type Type of API request: either "related" or "recommended"
   * @param string $tags Tags as comma delimited string, e.g. "product,page"
   * @param int $limit Limit for max results
   * @param string $exclude "true" if you want to exclude linked, "false" otherwise
   *
   * @return string The target URL
   */
  private function buildURL($item, $type, $tags, $limit, $exclude)
  {
    $targeturl = DE_BASE_URL;
    $targeturl .= $item; // Item
    $targeturl .= "/" . $type; // Type
    $targeturl .= "?tags=" . $tags; // Tags
    $targeturl .= "&maxresults=" . $limit; // Limit
    $targeturl .= "&excludeLinked=" . $exclude; // Exclude
    return $targeturl;
  }

  /**
   * Returns the cURL response given a target URL
   * @param string $targeturl The target URL for cURL
   *
   * @return string cURL Response
   */
  private function getCurlResponse($targeturl)
  {
    $ch = curl_init($targeturl);
    curl_setopt($ch, CURLOPT_POST, FALSE);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
    $response = curl_exec($ch);
    curl_close($ch);
    return $response;
  }

Answer 1

I wasn’t aware of curl_multi before your question, that’s a pretty bizarre (for PHP) interface.

It looks like there’s a Hello World example in the curl_multi_init documentation

// create both cURL resources
$ch1 = curl_init();
$ch2 = curl_init();

// set URL and other appropriate options
curl_setopt($ch1, CURLOPT_URL, "http://www.example.com/");
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch2, CURLOPT_URL, "http://www.php.net/");
curl_setopt($ch2, CURLOPT_HEADER, 0);

//create the multiple cURL handle
$mh = curl_multi_init();

//add the two handles
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);

$running=null;
//execute the handles
do {
    usleep(10000);
    curl_multi_exec($mh,$running);
} while ($running > 0);

//close the handles
curl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);
curl_multi_close($mh);

I’d start with that.