Question

I’m using RoR trying to search a simple form at my college using mechanize. The code works fine for searching google, but returns the search form in the results? I’m really confused. Any advice? Thanks!

ruby script/console
require 'mechanize'
agent = WWW::Mechanize.new
agent.get("https://www.owens.edu/cgi-bin/class.pl/")
agent.page.forms
form = agent.page.forms.last
form.occ_subject = "chm"
form.submit

Answer 1

I’ve solved it! When form.submit is being called, it is assuming the last button in form.buttons is the button to use. The last button in form.buttons is for the advanced form, hence the resulting page object being another form, albeit the more comprehensive advanced search form.

require 'mechanize'
agent = WWW::Mechanize.new
agent.get("https://www.owens.edu/cgi-bin/class.pl/")
agent.page.forms
form = agent.page.forms.last
form.occ_subject = "chm"
result = agent.submit(form, form.buttons.first)

result.parser.css('table.cs-table-settings tr.tbl-class-fill-b td font b').map { |v| v.text.strip }

=> ["Principles of Chemistry", "Principles of Chemistry", "Principles of Chemistry", "Principles of Chemistry", …]

Finally we get to the bottom of it! The HTML is horrible, so you will need to put your XPath hat on for this one! 🙂