Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I am working on a ROR application where I need to implement a crawler that crawls other sites and stores data in my database. For example suppose I want to crawl all deals from http://www.snapdeal.com and store them into my database. How to implement this using crawler?
There are couple of options depending upon your usecase.
I have used combination of Nokogiri and Mechanize for few of my projects and I think they are good options.