I currently have been assigned to create a web crawler to automate some reporting tasks I do. This web crawler would have to login with my credentials, search specific things in different fields (some in respect to the the current date), download CSVs that contain the data if there is any data available, parse the CSVs quickly to get a quick number count, create an email with the CSVs attached and send it.
I currently know C++ and Python very well, am in the process of learning C, but I was told that Ruby or Ruby on Rails was a great way to do this. Is Ruby on Rails solely for creating web apps, and if so, does my task fit the description of a web app, or can I just make a standalone program that runs and does it all?
I would like to know which language would be the easiest to code with (has easy to use modules), has a good library/module relative to these tasks. What would I need to take into account before undergoing this task? I have till the end of December to make this, and I only work here for around 12 hours per week (I’m a student, and this is for my internship). Is this feasible?
Thanks.
You already know Python, go with that. CSV parsing and mail sending are pretty trivial tasks and I assume you can figure those out with Google.
As for web crawling? Use Mechanize.