Question

I’ve written a nice import for my million row CSV that works quite nicely (using OPENROWSET BULK (I didn’t use BULK INSERT because I need to cross join with some other columns). The formatfile uses a comma as the Terminator.

Here is an example of the CSV I was using to develop:

Reference, Name, Street
1,Dave Smith, 1 Test Street
2,Sally SMith,1 Test Street

Once I’d got it working, someone reminded me that the data itself could have a comma in it, whoops!!!!

Reference, Name, Street
"1","Dave Smith", "1 Test Street"
"2","Sally Smith","1,Test Street" <-comma in street

How do people deal with a CSV with a comma in the data using FormatFiles? (Or do I just say that the file must be TAB delimited)?

Answer 1

If your field terminator can occur in the data, then it may be best to use TAB or PIPE delimiter (or whatever works for your data).

If a terminator character occurs within the data, it is interpreted as
a terminator, not as data, and the data after that character is
interpreted as belonging to the next field or record. Therefore,
choose your terminators carefully to make sure that they never appear
in your data

http://msdn.microsoft.com/en-us/library/ms191485.aspx