Question

I am trying to scrape a website. I am able to scrape data from that website. I am having trouble saving the data from the scrape to yaml file that I have included

My Code:

require 'rubygems'
require 'open-uri'
require 'hpricot'

article = []
     doc = open("http://www.cmegroup.com/trading/interest-rates/cleared-otc/irs.html"{|f| Hpricot(f) }

      (doc/"/html/body/div/div/div/div/table/").each do |article|
      puts "#{article.inner_html}"
       end

File.open('test.yaml', 'w') { |f|
f <<article.to_yaml
}

Answer 1

First you are missing a closing parenthesis for the open call (a ) right before the block starts).

When you add that you’ll notice that you’ll get a NoMethodError (undefined method 'to_yaml' for []:Array). To fix that you have to require 'yaml', which pulls in the monkey-patches for the Array class. After that you’ll notice that your yaml file is empty, because you never put anything into article. Here’s a fixed version:

require 'rubygems'
require 'open-uri'
require 'hpricot'
require 'yaml'

articles = []
url = "http://www.cmegroup.com/trading/interest-rates/cleared-otc/irs.html"
doc = open(url) {|f| Hpricot(f) }

  (doc/"/html/body/div/div/div/div/table/").each do |article|
    articles << article.inner_html
  end

File.open('test.yaml', 'w') { |f| f << articles.to_yaml }