Question

I’m sure this is a common problem, but I just can’t seem to find exactly the help I’m looking for. Apologies!

I’m trying to perform a bulk insert from a CSV file—the table I need to insert the data into has a few fields that cannot be null—like a row unique identifier field that is normally generated with newid(), and a few other fields such as the ID number of the user making the changes, etc…

Poking around online, I think I understand how to do a basic bulk import if the CSV file contains the same number of fields as the table, OR if it’s okay to leave missing fields null or default, but I haven’t been able to find somewhere that shows me how to assign some values from the CSV files, and to also predefine the other required table values. It would be extremely helpful if someone could show me how to do an import from a CSV where I can assign missing fields from variables within my stored procedure!

Any tips? Here is the code I’ve been trying to use to get this done.

BULK
INSERT tbl1_
FROM '\\...\importtest.txt'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
--Check the content of the table.
SELECT *
FROM tbl1_
GO
--Drop the table to clean up database.
SELECT *
FROM tbl1_
GO

Answer 1

Have you seen “Keeping Nulls or Using Default Values During Bulk Import” that applies to bcp and BULK INSERT?
I haven’t done this for a while (I’d load a simple staging table and enrich later in a separate flush step) but it looks like defaults can be kept when you use:

INSERT ... SELECT * FROM OPENROWSET(BULK...)

For missing columns, you need to use format files.

The two ideas are slightly different. Personally, I’d load a staging table and then copy from that so the CSV and staging table columns match. Then, I’d have a 2nd step where I load the “real” table. This allows you to verify data first, or to do some post-bcp processing.