I was looking for a way to retrieve Google search results, using RoR.
I could see that a while ago it used to be done only by scraping the HTML pages (Scruby, Nokogiri, Mecheanize…). I was wondering first, if it’s legit, second, I didn’t really undesrtand how can you rely on another site’s HTML and CSS – what would happen if it’s changed in the next day… Maybe I missed something there?
Then I saw that Google offers an API for doing it, but I couldn’t find any Rails plugin that handles it.
Does anybody know any?
Thanks :-).
There’s the googleajax gem, but it can only return up to 100 results.
Other than that, yes you can scrape google results, no it’s against their ToS (see 5.3), yes you still can do it a little and no one will notice or take action.
By the way, sites don’t normally change HTML markup often, and Google probably does it less often than average. Screen scraping is stable enough to be practically useful.