Question

I am using CURL to download 4 files from the url for which i have to authenticate myself. and i am authenticating myself with CURL only. But the problem is when i download data it only download 1KB of each file. I tried various methods given on SO. But none of them are working for me. My CURL-PHP code is

<?php

  require ('dbconfig/dbconfig.php'); //database connection.
  ini_set('display_errors', 1); 
  error_reporting(E_ALL);
  $username = "user";
  $password = "password";
  $url = "http://domain-name/contents/";

  global $ch;
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
  curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
  curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
  curl_setopt($ch, CURLOPT_HTTPHEADER, Array("Content-Type: text/xml"));
  curl_setopt($ch, CURLOPT_UNRESTRICTED_AUTH, 1);
  $output = curl_exec($ch);
  $r = time()-(24*60*60); 
  $dateit = date("Ymd", $r);

  $file1 = "aaa".$dateit.".tbz";
  $file2 = "aaa".$dateit.".tbz.md5";
  $file3 = "bbb".$dateit.".tbz";
  $file4 = "bbb".$dateit.".tbz.md5";

  $arr = array($file1, $file2, $file3, $file4);

  for($i = 0; $i <= 3; $i++) {

    $url = "http://domain-name/content/current/".$arr[$i];

    $writefn = function($ch, $chunk) { 
      static $data='';
      static $limit = 500; // 500 bytes, it's only a test

      $len = strlen($data) + strlen($chunk);
      if ($len >= $limit ) {
        $data .= substr($chunk, 0, $limit-strlen($data));
        echo strlen($data) , ' ', $data;
        return -1;
      }
      $data .= $chunk;
      return strlen($chunk);
    };

    $ch = curl_init();
    $fp = fopen("adminArea/folder/".$arr[$i], 'w+');

    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RANGE, '0-500');
    curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1);
    curl_setopt($ch, CURLOPT_WRITEFUNCTION, $writefn);
    curl_setopt($ch, CURLOPT_FILE, $fp);

    curl_exec ($ch);

  }

  fclose($fp);
  curl_close($ch);

?>

Firstly i am authenticating myself then in for loop for all four i am using curl to download data. I got this curl code too from SO. Tried a lot of ways to do the same.

Any help or idea will be highly appreciated.

UPDATE

code worked for me. I added curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); both times as firstly it will authenticate and return response and then download file for me from other url.

 // Set cURL options
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_UNRESTRICTED_AUTH, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1);

// Do it
curl_exec ($ch);

curl_setopt($ch, CURLOPT_URL, $urlbase.$file);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$fp = fopen("adminArea/folder/$file", 'w+');
curl_setopt($ch, CURLOPT_FILE, $fp);

// Do it
curl_exec ($ch);

// Close pointers    
fclose($fp);
curl_close($ch);

Answer 1

I think the problem is most likely to do with the callback write function. I can;t quite work out exactly what this was supposed to achieve as it is, but it certainly isn’t doing you any favours. Indeed, it seems to me like there’s a lot of code there that is not required for what you describe.

Try this:

<?php

  // Database connection - do you need this here? You never use it...
  require ('dbconfig/dbconfig.php');

  // While developing
  ini_set('display_errors', 1); 
  error_reporting(E_ALL);

  // The details of the destination
  $username = "user";
  $password = "password";
  $urlbase = "http://domain-name/content/current/";

  // Create the file list
  $dateit = date("Ymd", strtotime('-1 day'));
  $files = array("aaa$dateit.tbz", "aaa$dateit.tbz.md5", "bbb$dateit.tbz", "bbb$dateit.tbz.md5");

  // Loop the file list
  foreach ($files as $file) {

    // Create pointers
    $ch = curl_init($urlbase.$file);
    $fp = fopen("adminArea/folder/$file", 'w+');

    // Set cURL options
    curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
    curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
    curl_setopt($ch, CURLOPT_UNRESTRICTED_AUTH, 1);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1);
    curl_setopt($ch, CURLOPT_FILE, $fp);

    // Do it
    curl_exec ($ch);

    // Close pointers    
    fclose($fp);
    curl_close($ch);

  }

?>