I am teaching myself Ruby on Rails. I would like to make website that whenever someone visits it, will scrape another website and display some data. Is this possible?
Share
Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I am teaching myself Ruby on Rails. I would like to make website that whenever someone visits it, will scrape another website and display some data. Is this possible?
Yes, it’s possible.
Just remember one things: Don’t crawl data within your controller action. Crawling data might be a long running process. The target website might be slow or down, and it will block your entire website. You should use some cron job or job queue to crawl data, and store in your database. The rails app gets data from database, not from the other website directly.