Question

I am very new to Ruby and I am currently working on site-scraping using Nokogiri to practice. I would like to scrape the details from ‘deals’ from a random group-buying site. I have been able to successfully scrape a site but I am having problems in parsing the output. I tried the solutions suggested in here and also using regex. So far, I have failed.

I am trying to parse the following title/description from this page:

Frosty Frappes starting at P100 for P200 worth at Café Tavolo – up to 55% off

This is what I got:

FrostyFrappes starting at P100 for P200 worth at Caf Tavolo  up to 55% off

Here are the snippets in my code:

require 'uri'
require 'nokogiri'

html = open(url)
doc = Nokogiri::HTML(html.read)
doc.encoding = "utf-8"
title = doc.at_xpath('/html/body/div/div[9]/div[2]/div/div/div/h1/a')
puts title.content.to_s.strip.gsub(/[^0-9a-z%&!\n\/(). ]/i, '')

Please do tell me if I missed something out. Thank you.

Answer 1

Your xpath is too rigid and your regex is removing chars you want to keep. Here’s how I would do it:

title = doc.at('div#contentDealTitle h1 a').text.strip.gsub(/\s+/,' ')

That says take the text from the first a tag that comes after div#contentDealTitle and h1, strip it (remove leading and trailing spaces) and replace all sequences of 1 or more whitespace char with a single space.