Question

I have a question – I’m using CURL and PROXY. I’m getting list with proxy addresses from paid web service. The list looks like this (example):

123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234

Let’s say, that the list mentioned above is available here: http://example.com/list_with_the_proxy

My PHP code looks like this:

<?php
$url = 'http://connect.to.another.example.net/'; //I want to open this url using one of the proxy address from the list

$proxy_url = 'http://address.of.proxy.example.org';

function RandomLine($filename) { 
    $lines = file($filename) ; 
    return $lines[array_rand($lines)] ; 
} 

$random_proxy = RandomLine($proxy_url); 

$ch = curl_init(); 
curl_setopt($ch, CURLOPT_URL, $url); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1); 
curl_setopt($ch, CURLOPT_PROXY, $random_proxy); 
$data = curl_exec($ch); 
curl_close($ch); 
echo $data;

?>

I have noticed some problem – sometimes some proxy is unavailable, and the site is loading & loading & loading … Is there any possibility, to reload the script and get new random proxy, if the site is loading too long ? For example if proxy is not responding in 5 seconds, script will load again ?

Answer 1

Look into CURLOPT_TIMEOUT, reload in every case when you don’t get any data in a reasonable time. Also are you sure that the data from the proxy list site is coming through correctly and its not hanging because of that?