Question

I am trying to process a CSV file that can either be generated with CF or LF as an EOL marker. When I try to read the file with

infile = File.open('my.csv','r')
while line = infile.gets
...

The entire 20MB file is read in as one line.

How can I detect and handle properly?

TIA

Answer 1

I would slurp the file, normalize the input, and then feed it to CSV:

raw = File.open('my.csv','rb',&:read).gsub("\r\n","\n")
CSV.parse(raw) do |row|
  # use row here...
end

The above uses File.open instead of IO.read due to slow file reads on Windows Ruby.