I am getting a CSV from a client, I need to import that file

Question

0

Asked: June 13, 20262026-06-13T12:22:03+00:00 2026-06-13T12:22:03+00:00

I am getting a CSV from a client, I need to import that file

0

I am getting a CSV from a client, I need to import that file to the database. I’ve been through countless iterations, part of the solution tests infinitum. My problem comes when I need the full solution.

The CSV has 14 fields, the tempdb database has 15 fields (last one is the identity column). The data arrives without an identity and I need a row unique number as per the db design.

I’ve obscured the test data shown, but it mimics the data ok.

T2012-DAT;09-01-2012;09-01-2012;1;910,91;12;TST;4,55;200,2;6;;;;7,5
T2012-DAT;10-01-2012;10-01-2012;1;910,91;12;TST;4,55;200,2;6;;;;7,5
T2012-DAT;11-01-2012;11-01-2012;1;910,91;12;TST;4,55;200,2;6;;;;7,5
T2012-DAT;12-01-2012;12-01-2012;1;910,91;12;TST;4,55;200,2;6;;;;7,5
T2012-DAT;13-01-2012;13-01-2012;1;910,91;12;TST;4,55;200,2;6;;;;7
T2012-DAT;16-01-2012;16-01-2012;1;910,91;12;TST;4,55;200,2;6;;;;7,5

The temporary table I create in the tempdb. You will notice it has 15 fields, last one being the needed identity.

CREATE TABLE BudgetImport(
    sBudgetName varchar(20) COLLATE Danish_Norwegian_CI_AS  
    , dStartDate varchar(12) COLLATE Danish_Norwegian_CI_AS  
    , dEndDate varchar(12) COLLATE Danish_Norwegian_CI_AS   
    , prCode int   
    , decTotal varchar(20) COLLATE Danish_Norwegian_CI_AS
    , sRefTimeTypeID varchar(10) COLLATE Danish_Norwegian_CI_AS  
    , sRefEmployeeID varchar(10) COLLATE Danish_Norwegian_CI_AS  
    , decHours varchar(20) COLLATE Danish_Norwegian_CI_AS  
    , decRate varchar(20) COLLATE Danish_Norwegian_CI_AS
    , sDepartmentID varchar(10) COLLATE Danish_Norwegian_CI_AS NULL  
    , sCentre varchar(10) COLLATE Danish_Norwegian_CI_AS NULL  
    , sPurpose varchar(10) COLLATE Danish_Norwegian_CI_AS NULL  
    , sProjectID varchar(10) COLLATE Danish_Norwegian_CI_AS NULL  
    , decNormHours varchar(20) COLLATE Danish_Norwegian_CI_AS  
    --, iRowNumber int identity(500000,1)  
)

GO

 -- import data by csv
 BULK INSERT BudgetImport
 FROM 'D:\budgetposter.csv'
 WITH
 (
    fieldterminator = ';'
    , rowterminator = '\r\n'
    , codepage = '1252'
 )

When I incl the identity I get this error:

Msg 4866, Level 16, State 1, Line 3
The bulk load failed. The column is too long in the data file for row 1, column 15. Verify that the field terminator and row terminator are specified correctly.

If I exclude it, the error changes to:

Msg 4866, Level 16, State 1, Line 3
The bulk load failed. The column is too long in the data file for row 1, column 14. Verify that the field terminator and row terminator are specified correctly.

If I changes the rowterminator = '\n' it comes through, but I am missing the identity.

If I then incl the identity the error becomes:

Msg 4864, Level 16, State 1, Line 3
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 15 (iRowNumber).

When I’m testing I’m excluding the INSERT INTO dbo."the-real-datatable" and just doing the SELECT FROM BudgetImport alone. Hence I avoid inserting another 16000 rows into the real-datatable.

If you pay attention and know the hungarian notation, you will notice I am using varchars even though datatime or decimal might be the correct form. I am doing this after about 4-6 hours testing and endless supply of headaches and torn hair. Varchar is the KISS solution, I do a CONVERT() later.

My whole problem resolves around the last field in the row.
– I’ve tried adding 1 and 2 ; after the decNormHours, I’ve tried adding empty values (; ; or ;1; or ;n;).
– I’ve tried with a formatfile.xml – again no cake. I had more success without a formatfile than with it.
– I’ve tried changing to UK number notation , to . = more success with the DK notation.
– I have tried every single trick I can come up with to no avail.

Yes there is no identity field in the CSV, that is ok. You just use the default auto identity(500000,1) I have described in the table creation. Oh the 500000 is an approx row id I’ve reached now, it will increase over time.

Please, what need I try next to make this good?

EDIT:
Using the format file

$lt;? xml version="1.0 " ? $gt;
$lt;BCPFORMAT xmlns="http://schemas.microsoft.com/sqlserver/2004/bulkload/format" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"$gt;
 $lt;RECORD$gt;
  FIELD ID="1" xsi:type="NCharTerm" TERMINATOR=";" COLLATION="SQL_Latin1_General_CP1_CI_AS"/
  FIELD ID="2" xsi:type="CharFixed" LENGTH="9" COLLATION="SQL_Latin1_General_CP1_CI_AS"/
  FIELD ID="3" xsi:type="CharFixed" LENGTH="9" COLLATION="SQL_Latin1_General_CP1_CI_AS"/
  FIELD ID="4" xsi:type="NCharTerm" TERMINATOR=";"/
  FIELD ID="5" xsi:type="NCharTerm" TERMINATOR=";"/
  FIELD ID="6" xsi:type="NCharTerm" TERMINATOR=";" MAX_LENGTH="20" COLLATION="SQL_Latin1_General_CP1_CI_AS"/
  FIELD ID="7" xsi:type="NCharTerm" TERMINATOR=";" MAX_LENGTH="20" COLLATION="SQL_Latin1_General_CP1_CI_AS"/
  FIELD ID="8" xsi:type="NCharTerm" TERMINATOR=";"/
  FIELD ID="9" xsi:type="NCharTerm" TERMINATOR=";"/
  FIELD ID="10" xsi:type="NCharTerm" TERMINATOR=";" MAX_LENGTH="20" COLLATION="SQL_Latin1_General_CP1_CI_AS"/
  FIELD ID="11" xsi:type="NCharTerm" TERMINATOR=";" MAX_LENGTH="50" COLLATION="SQL_Latin1_General_CP1_CI_AS"/
  FIELD ID="12" xsi:type="NCharTerm" TERMINATOR=";" MAX_LENGTH="50" COLLATION="SQL_Latin1_General_CP1_CI_AS"/
  FIELD ID="13" xsi:type="NCharTerm" TERMINATOR=";" MAX_LENGTH="20" COLLATION="SQL_Latin1_General_CP1_CI_AS"/
  FIELD ID="14" xsi:type="NCharTerm" TERMINATOR=";"/
 /RECORD
 ROW
  COLUMN SOURCE="1" NAME="sBudgetName" xsi:type="SQLNVARCHAR" LENGTH="20" /
  COLUMN SOURCE="2" NAME="dStartDate" xsi:type="SQLDATETIME"/
  COLUMN SOURCE="3" NAME="dEndDate" xsi:type="SQLDATETIME"/
  COLUMN SOURCE="4" NAME="prCode" xsi:type="SQLSMALLINT"/
  COLUMN SOURCE="5" NAME="decTotal" xsi:type="SQLDECIMAL"/
  COLUMN SOURCE="6" NAME="sRefTimeTypeID" xsi:type="SQLNVARCHAR" LENGTH="10"/
  COLUMN SOURCE="7" NAME="sRefEmployeeID" xsi:type="SQLNVARCHAR" LENGTH="10"/
  COLUMN SOURCE="8" NAME="decHours" xsi:type="SQLDECIMAL"/
  COLUMN SOURCE="9" NAME="decRate" xsi:type="SQLDECIMAL"/
  COLUMN SOURCE="10" NAME="sDepartmentID" xsi:type="SQLNVARCHAR" LENGTH="10"/
  COLUMN SOURCE="11" NAME="Centre" xsi:type="SQLNVARCHAR" LENGTH="10"/
  COLUMN SOURCE="12" NAME="Purpose" xsi:type="SQLNVARCHAR" LENGTH="10"/
  COLUMN SOURCE="13" NAME="sRefProjectID" xsi:type="SQLNVARCHAR" LENGTH="10"/
  COLUMN SOURCE="14" NAME="decNormHours" xsi:type="SQLDECIMAL"/
/ROW

Puts me back at making the first field work out right. So from fighting getting the final detail right, I end up at fighting for making it work in the first place / at doing anything at all.
Error:

Msg 4863, Level 16, State 1, Line 3
Bulk load data conversion error (truncation) for row 1, column 1 (sBudgetName).

note: hmm seems SO doesn’t like xml. How odd.

Report

Leave an answer
Cancel reply

You must login to add an answer.

Need An Account,

1 Answer

Editorial Team · Answer 1 · 2026-06-13T12:22:04+00:00

This is more a workaround than a solution but since you face a problem with number of columns, why don’t you add the index column after ?

You could do something like this :

Import your CSV into your temp table with same number of columns

Then,

ALTER TABLE BudgetImport
    ADD Id  INTEGER
GO

WITH Cte
AS
(
    SELECT *
    , ROW_NUMBER() OVER(ORDER BY [**column of your choice**] DESC) AS RowNumber
    FROM BudgetImport
)
UPDATE Cte
SET Id = RowNumber
GO

ALTER TABLE BudgetImport
ALTER COLUMN Id INTEGER NOT NULL 
GO

ALTER TABLE BudgetImport
ADD PRIMARY KEY (Id)
GO

Sign Up

Sign In

Forgot Password

The Archive Base Latest Questions

I am getting a CSV from a client, I need to import that file

Leave an answerCancel reply

1 Answer

Leave an answer
Cancel reply