Question

I am using Mechanize gem to parse html content.

Firing this on terminal,

 agent.get("http://www.example.com/").search(".sidebar-deal-excerpt").first

gives me this:

 <Nokogiri::XML::Element:0x3fcdb3add700 name="div" attributes=[#<Nokogiri::XML::Attr:0x3fcdb3add570 name="class" value="sidebar-deal-excerpt">] children=[#<Nokogiri::XML::Text:0x3fcdb3add0ac "\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t">, #<Nokogiri::XML::Element:0x3fcdb3adcfd0 name="a" attributes=[#<Nokogiri::XML::Attr:0x3fcdb3adcf6c name="href" value="http://www.example.com/getaway-8774-yatra-september-deal">] children=[#<Nokogiri::XML::Text:0x3fcdb3adcaa8 " ">, #<Nokogiri::XML::Element:0x3fcdb3adc9cc name="span" attributes=[#<Nokogiri::XML::Attr:0x3fcdb3adc968 name="style" value="color: #000">] children=[#<Nokogiri::XML::Text:0x3fcdb3dd6b50 " international holiday package bookings at Yatra.com ">]>, #<Nokogiri::XML::Text:0x3fcdb3dd5f5c " ">]>, #<Nokogiri::XML::Text:0x3fcdb3dd5a98 "\r\n\t\t\t\t\t\t\t\t\t\t\t\t">]> 

Firing this on terminal

agent.get("http://www.example.com/").search(".sidebar-deal-excerpt").first.children

gives me this:

 <Nokogiri::XML::Text:0x3fcdb399c3dc "\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t">, #<Nokogiri::XML::Element:0x3fcdb399c300 name="a" attributes=[#<Nokogiri::XML::Attr:0x3fcdb399c29c name="href" value="http://www.example.com/getaway-8774-yatra-september-deal">] children=[#<Nokogiri::XML::Text:0x3fcdb399bdd8 " ">, #<Nokogiri::XML::Element:0x3fcdb399bcfc name="span" attributes=[#<Nokogiri::XML::Attr:0x3fcdb399bc98 name="style" value="color: #000">] children=[#<Nokogiri::XML::Text:0x3fcdb399b7d4 "international holiday package bookings at Yatra.com ">]>, #<Nokogiri::XML::Text:0x3fcdb3dd8770 " ">]>, #<Nokogiri::XML::Text:0x3fcdb3dd6df8 "\r\n\t\t\t\t\t\t\t\t\t\t\t\t">]

The thing is, I want to fetch url from this. I want to get this attribute value

value="http://www.example.com/getaway-8774-yatra-september-deal"

ie

   http://www.example.com/getaway-8774-yatra-september-deal

How can I parse this?

Answer 1

How about: (untested)

agent = Mechanize.new
page = agent.get("http://www.snapdeal.com/")
page = agent.get("http://www.snapdeal.com/") # yeah, you've got to call it twice for this site
urls = page.search(".sidebar-deal-excerpt a").collect{|a| a.attributes["href"].value }