Question

I am importing csv to db using bulk insert.
It is the comma delimited csv file. No text qualifiers for all fields.

But some fields may have comma as part of the data.
for eg, ADDRESS field value. Those values are surronded with double quotes.
Those double quotes appear only if the field value has comma in it otherwise values are not surronded with double quotes.
So in some rows ADDRESS values are surronded with double-quotes, but in other rows they are not.
Is there a way to specify the text-qualifier in the bulk insert command?

I tried bulk insert with format file option.

BULK INSERT Test_Imported FROM 'C:\test.csv' 
WITH (FIRSTROW=0,FIELDTERMINATOR = ',',ROWTERMINATOR = '\n',FORMATFILE = 'C:\test.Fmt')

but there is no way i can mention the double quotes as optional text qualifiers in the format file.

PS:
this function is actually a part of the bigger module, which is written in c#.
bulk insert command is called from c#.

The csv file is coming by email from another automated system. i have no control over the format
of the csv file.There are around 150 columns.
In average 12000 rows are coming in each csv file.
Forgot to spcify the DB. It is SQL server 2005.

Answer 1

Unfortunately, you’ll have to pre-process the file to make it consistent.
SQL bulk operations split the string on the field delimiter.

Some options:

• Process in c# to change commas not surrounded by quotes to pipe (|)
• Break the file in 2: ” and non-” files. This works only if the same field has “

You say you have no control over the format, but what you have is unusable…