Question

I have a comma delimited CSV file being loaded into a String value generated as Part of an XSLT Transformation in C# – (CLR assembly, console Application).

I need to get this C# app to output this CSV file into a Database table, 1 line at a time. and as I am rather inexperienced with C#, I have no idea how best to achieve this!

In SQL I managed it by the following SQL Statement:

INSERT INTO CsvData 
    (ID, sFilename, iLineCount, sData, dDate)
SELECT @ID, @Filename, id, val, CAST(getdate() as smalldatetime) 
  FROM dbo.split(@CSVFile,char(13))

The dbo.split function takes the @CSVFile and it’s Delimeter (char(13) in this case) and returns a Table with 1 row per line in the CSVFile (id=identity, val=Linedata), which is then used to populate the CsvData table.

I cannot pass the @CSVfile as a parameter to an SQL StroedProc as it can get VERY large, so want to keep it all enclosed in the C# code.

How would i best achieve this in .NET?

Answer 1

You can use SqlBulkCopy class from .NET SqlClient provider. Check this MSDN article for its usage. It provides good performance regarding bulk inserts. However, you must firstly read your data from CSV in a structured format, since SqlBulkCopy requires a DataTable or an IDataReader for it to work. You have 2 options:

1. Load your entire CSV file into a DataTable object, which may not be the best solution if your CSV file is very big
2. Create a CsvDataReader as a read-only, forward-only cursor for your CSV files. You can find some implementations on web.

Since you’re saying that your CSV is being loaded into a String value, than if wouldn’t be a problem to fill a DataTable with data. Use this DataTable as an argument to SqlBulkCopy.WriteToServer method.