Question

First and foremost, I am writing a python script to automate purchasing of certain domains from dreamhost.com. I first go to the website’s panel where users can do pretty much anything the site has to offer. Here is the code:

br = mechanize.Browser()
br.open("https://panel.dreamhost.com/index.cgi?tree=domain.registration&")
br.select_form(name="a")
br["username"]="my_username_here"
br["password"]="my_password_here"
br.submit()

This code words perfectly and logs me into the panel. From here, there is a textbox to enter your domain and a multi-select to choose your extension – when this form is submitted, it checks for availability.

I take a similar approach to the success above and perform the following:

br.select_form(name="f")
br["Pick Domain Name"] = "domainiwanttobuy"    #textbox
br["tld"] = [extension list]                   #select from dropdown
br.submit()

The resulting HTML after this form has been submitted shows that the dropdown changes appropriately, but the text field is blank and I get an error from dreamhost that says, “Error: Please enter a domain name above.”

So the input field isn’t working – I looked at the corresponding page source HTML and here it is:

<input name="fqdn" id="fqdn" class="text reg_domain_input" name="Pick Domain Name" value="mynewdomain" onChange="tradeMarkCheck(this.id)">

First of all it’s suspect that there are 2 name elements. I cannot use the name=”fqdn” or it says that such a control doesn’t exist. Secondly, I see that a javascript function is called when the value changes. I’m aware that mechanize cannot handle Javascript, but I don’t believe that the function does anything of value. So, I don’t think it’s causing the problem. Here is the corresponding javascript function:

<script type="text/javascript">
function tradeMarkCheck(x) {
    var domain = document.getElementById(x).value;
    var bad = /(dream.*host|host.*dream)/;

    if (bad.test(domain)) {
        alert("NOTE: Domains containing 'DreamHost' in them are not eligible for use with DreamHost Rewards.  Also, please ensure that your site/domain does not dilute or infringe upon the 'DreamHost' trademark (if you have any questions as to whether a use is legal or not, contact us at [e-mail]).");
    }
}
</script>

I’ve spent hours debugging this and trying to get a workaround. Any ideas would be greatly appreciated. Thanks in advance!

Answer 1

I haven’t looked at the site but I strongly suspect that some javascript is doing something that you’re not doing with mechanize. The standard answer in these cases is to use Selenium Webdriver to automate a browser (it has a Python API).

Edit:
Selenium Webdriver has HtmlUnit headless browser that supports javascript a little, but it’s far from full javascript support. PhantomJS might be able to help you out, it’s a headless version of Webkit.