Question

I’m trying to use cURL (in a bash script) to load a page on my ISPs site containing my monthly data usage.

Here’s the relevent form from the page:

<form method="post" id="Login" name="Login" action="https://signon.bigpond.com/login" class="form">
    <input type="hidden" name="goto" value="https://my.bigpond.com/mybigpond/default.do?ref=Net-Header-MyBigPond1"/>

    <div class="loginModule roundify">
        <div class="loginForm">

            <div class="formRow error">
                <label for="userName">
                    Username<span class="reqField">*</span>
                </label>
                <input type="text" tabindex="1" id="username" name="username" size="30" maxlength="200" onkeypress="EnterKeyPress(event)" value=""/>
            </div>

            <div class="formRow error">
                <label for="password">
                    Password<span class="reqField">*</span>
                </label>
                <input type="password" tabindex="2" id="password" name="password" size="30" maxlength="50" onkeypress="EnterKeyPress(event)"/>                                     
            </div>

           <div class="buttons">
                <input class="submit roundify" type="submit" value="Log in" onclick="setCookieForUser();this.disabled=true;document.forms['Login'].submit();" />
            </div>
        </div>
    </div>
</form>

Note that the submit button doesn’t have a name.

The function called when text is entered into the form:

function EnterKeyPress(e) {
    if (!e) var e = window.event;

                if (e.keyCode) code = e.keyCode;
                else if (e.which) code = e.which;

                if ( code == 13 && document.getElementById("username").value.length > 0 && document.getElementById("password").value.length > 0 ) {
                    document.forms['Login'].submit();
                    e.returnValue = false;
                }
}

The cURL command line:

 $ curl --user-agent "Mozilla/5.0" --verbose --location --cookie-jar "/tmp/bigpond.cookies" --cookie "/tmp/bigpond.cookies" -o "/tmp/bigpond.html" --data-urlencode "username=MY_USERNAME&password=MY_PASSWORD" https://signon.bigpond.com/login?goto=https://my.bigpond.com/mybigpond/default.do?ref=Net-Header-MyBigPond1

However, the saved html file (/tmp/bigpond.html) is just the logon page still, with text added saying I forgot to enter my username and password. Why is cURL not sending the POST data or what am I doing wrong?

Update: Answered my own question. See below for solution.

Answer 1

The problem was passing the POST data to cURL in one argument was causing the data to be mangled when cURL did the URL encoding. The correct usage is to pass each field as a separate argument like so:

curl --data-urlencode "username=XXXXXX" --data-urlencode "password=XXXXXX" ...