Question

Hi!

I’m trying automatize image uploading to a site, witch is using database to keep track of the files. The user interface is… well, pretty crappy, and I have a lot of files. I have no access to the database, so I had to find a workaround for that and I found that I could easily mimic the site’s uploading mechanism with cURL.

I examined the POST data sent by the browser with Mozilla’s Firebug, and the source code of the site’s upload form. I was able to determine the required POST fields to be sent to the server, so I’ve put a little cURL command together in PHP.

I’m using Debian GNU/Linux, and PHP with a version number of 5.3.20-1.
The relating code looks like this:

$strfilepath = '@' .  $item->getPathname();  // get next file from directory (directory iterator)
    
$postdata = array();

$postdata['upload'] = 1;
$postdata['category_id'] = 43;
$postdata['imgtoupload'] = $strfilepath;
$postdata['actie'] = 1;
        
curl_setopt($somecurl, CURLOPT_URL, "http://some.site.at.somwhere.com/image_up.php");
curl_setopt($somecurl, CURLOPT_POST, true);
curl_setopt($somecurl, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($somecurl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($somecurl, CURLOPT_NOPROGRESS, false);
curl_setopt($somecurl, CURLOPT_VERBOSE, true); // FTW!
        
$response = curl_exec($somecurl);

I want to compare the browser’s POST command to my "artifical" http request, so
my exact question is how to get the POST (or any other HTTP) request generated by cURL? (Like Firebug can do.)
The only thing I can do with curl is to enable CURLOPT_VERBOSE, but it gives back header data only.

Examining the return of the curl_getinfo() and the command line listings I found that the server gives an 302 HTTP warn, and it doesn’t upload any files:

Connection #0 to host some.site.at.somwhere.com left intact

Probably there’s some problem with authentication, but I want to make sure that the code sending the appropriate data for the server.

Thanks in advance!
Sincerely: G.

EDIT (13.01.09):

Ha! I’ve made it!
The issue was that cURL somehow overwritten the type of the sent data. It marked the image as application/octet-stream instead of image/jpeg.
I explicitly specified the type in the "file-to-be-sent" field, witch solved the problem:

 $strfilepath = '@' .  $item->getPathname() . ";type=image/jpeg" ;

It works as it should. Thanks for the help!

Answer 1

The easiest way is to use ngrep from the command line:

ngrep -W byline host some.site.at.somwhere.com and port 80

Which will dump all of the traffic to/from some.site.at.somwhere.com on port 80 to stdout. You can also have the output written to a pcap file with the -O flag:

ngrep -O http-traffic.pcap -W byline host some.site.at.somwhere.com and port 80

Some distributions don’t package ngrep in their main repositories, but tcpdump is a good alternative in that case and the syntax is similar:

tcpdump -A -s 1600 host some.site.at.somwhere.com and port 80

And to send that to a file, instead:

tcpdump -w http-traffic.pcap -s 1600 host some.site.at.somwhere.com and port 80

Note that the expression host some.site.at.somwhere.com and port 80 is the same regardless of which tool you use. You can read up on that with man pcap-filter.