Question

I have written a ruby script to screen scrape something using the ‘open-uri’ and ‘hpricot’ gems – everything works great so far.

But now I have to screen scrape something which is returned after a form is submitted via a javascript function (called by an ‘onchange’ event handler from a drop-down menu):

function submit_form() {
  document.list.action="/some/sort/of/path";
  document.list.submit();
}

AFAIK, open-uri lets you submit only GET requests. And if I’m not mistaken, a POST request would be needed here.

So my question is: what do I need to install and to ‘require’ and how would the ruby code then look like (to make that POST request) – sorry, I’m still pretty much of a n00b…

Thank you very much for your help!

Tom

Answer 1

I think you definitely should use Mechanize. It provides a nifty interface to interact with remote pages, forms on them, and so forth (see this example).