Question

I am building a small application in RoR that has a form asking for a URL. Once the URL has been filled in and submit button is pressed I have downloaded a web-scraping plugin scrAPI(which is working fine) which gets the of URL and creates a record in db with title.

My issue right now is that I am able to make the whole thing work if the URL is valid and scrAPI is able to process it. If a URL entered does not work it gives this “Scraper::Reader::HTTPInvalidURLError” which is expected, but my knowledge of working in Model is preventing me from handing that error in a correct manner.

Controller:

#controller
class ArticleController < ApplicationController

  def savearticle

    @newarticle = params[:newarticle]

    @link = @newarticle["link"]
    @id   = @newarticle["id"]
    Article.getlink(@link)

    success = Article.find(:last).update_attributes( params[:newarticle] )

    if success
      render :partial => 'home/articlesuccess'
    else 
      render :partial => 'home/articlebad'
    end

  end


end

# model
require 'scrapi'

class Article < ActiveRecord::Base

  attr_accessor :getlink

  def self.getlink(link)

    scraper = Scraper.define do
      process "title", :title => :text
      result :title
    end  

    uri = URI.parse(link)

    Article.create(:title => scraper.scrape(uri))

  end


end

How to:

1) Handle the Scraper::Reader::HTTPInvalidURLError properly, so text could be returned to view with proper error.

2) I would also like to know how I can return ‘uri’ from model and use it in the controller or view.

3) Also, I would like to return the ID of the Article created in Model so I can use that in the controller instead of doing find(:last) which seems like bad practice.

Answer 1

Something like…

class ApplicationController < ActionController::Base
  rescue_from 'Scraper::Reader::HTTPInvalidURLError', :with => :invalid_scrape_url


 private
   def invalid_scrape_url
      flash[:error] = 'The URL for scraping is invalid.'
      render :template => 'pages/invalid_scrape_url'
   end
end

rescue_from is what you need.

That’s 1)

for 2) You could just use @uri but personally I’d create a new model called Scrape and then you can retrieve each Scrape that is attempted.

for 3) I’m not quite sure of the question but

@article = Article.create(:title => scraper.scrape(uri))

then
@article.id

Hope that helps!