I’m building a website that is going to import products from several different sources. I will start with some XML import and later I might have to implement CSV import as well. It will fetch the XML from a URL and then check each post and see if the product already exists, if it does, check the price and update it if it has changed. I have heard good things about Nokogiri and will use that for the XML parsing. I will use Rails 3.1.1.
I am pretty new to Rails and have some questions. I’m trying to build this app for learning Rails and also try some TDD.
Should I build the import scripts as rake tasks? Is that a good approach?
How do I run these periodically then? Do I set up a cronjob as I do in PHP?
The XML sources will differ some in structure, element names and so on so I guess I will have to create an import script for each source. Is this a good approach or can I somehow create a more generic import script and somehow map the different elements to the correct column in my database?
Lots of questions… I hope someone can give me some tips and kick me in the right direction.
Thanks,
Linus
Lots of questions, I’ll try to answer some of them.
Nokogiri is a good choice.
Yes, you can run scripts as cron jobs, or you can set up a regular rails daemon (google it) there are lots of options available, depending on how often you want it to run, or how you’d like to set it up. I’ve created these things as a rake task then set cron to just run the rake task on a regular schedule. If you’re comfortable with rake and cron, this is a good idea.
yes, you will probably be able to find ways to combine scripts for similar XML. Your rake task will be in ruby – there’s nothing stopping you from having it test the XML for what kind it is, then call the appropriate method in a module in a library that you write to extract the data – then have a common method for creating a new object in your db with that data.