Question

Newbie to ruby this is my first project I am using the FasterCSV Gem and absolutely love the speed.

I have found one problem that when a field is too long it throws up an error below:

/Library/Ruby/Gems/1.8/gems/fastercsv-1.5.4/lib/faster_csv.rb:1641:in `shift’: FasterCSV::MalformedCSVError (FasterCSV::MalformedCSVError)

The error is on the second line first line is read correctly 🙂

“1013197145”,”PSION HU6020 Hand Strap”,”1″,”1″,”10″

“1013197542”,”HP ProBook NA920EA_KT931AT 39.6 cm (15.6″) Notebook – Intel Core 2 Duo T6570 2.10 GHz – 1366 x 768 WXGA Display – 2 GB RAM – 250 GB HDD – DVD-Writer LightScribe – Intel GMA 4500MHD Graphics Card – Bluetooth – Webcam – Genuine Windows Vista Business – 5 Hour Battery – HDMI”,”1″,”1″,”10″

here is my code:

FasterCSV.foreach(file_path, {:col_sep=> colsep, :quote_char => char}) do |row|

   my.query("INSERT INTO product_prices (partno,costpriceexvat,sku, supplierid) VALUES ('#{row[7]}', '#{row[8]}', '#{row[3]}','1')")

end

any help would be much appreciated.

Answer 1

I suspect this has nothing to do whatsoever with FasterCSV choking on long strings and everything with the inch character in the “(15.6″)” part of your item description – as it is identical to the quote, it terminates the CSV value, thus making for a malformed CSV line. FasterCSV is designed to be strict about this kind of things.