G’day guys,
I’m using fasterCSV and a rake script to parse a csv with about 30 columns into my rails db for a ‘Trade’ item. The script works fine when all of the values are set to strings, but when I change it to a decimal, int or other value, everything goes to hell.
Wondering if fasterCSV has built in int etc parsing or whether I’ll have to manage these within my model.
Basically, I’m given a giant amount of trades data, need to import it, and then need to provide feedback with say the average trade volume, the times, etc. I understand I can do that all with the wonderful records provided to me by activeRecord but wondered if there was an easier way to populate a rather large Database with a given CSV?
Several of the fields don’t have values for certain rows, fasterCSV seems to work perfectly when they’re all strings, but not when I try to get decimal or other.
Have you tried to use AR Extensions for bulk import? You get impressive performance improvements when you are inserting 1000’s of rows to DB. Visit their website for more details.
Refer to these examples for more information
Usage Example 1
Usage Example 2
Usage Example 3