Question

Hi I’m trying to use curl from the command line to submit a form and save the file that the form forces you to download normally when you use a browser. The form is on this page:

http://dove.cccbr.org.uk/downloads.php

Its the dove.txt file I want to download. As you can see in a browser you check the copyright notice check box and then click the dove.txt button and the file starts downloading.

Here’s how I’ve tried to do it with curl:

curl -d "copyright=agree&download=dove.txt" -o dove.txt http://dove.cccbr.org.uk/downloads.php

This saves a file but it just has an error message in it

DOCTYPE HTML PUBLIC “-//IETF//DTD HTML 2.0//EN”>
tml>
itle>406 Not Acceptable
head>
1>Not Acceptable

An appropriate representation of the requested resource /downloads.php could not be found
on this server.

Additionally, a 404 Not Found
ror was encountered while trying to use an ErrorDocument to handle the request.

body>

I’m sure this is very simple for someone who knows how

Answer 1

This specific site blocks the curl user agent string. You can spoof another user agent using the -A option, e.g.:

curl -A 'Mozilla/5.0' …