Question

I have experience using XML format files created with BCP command line and then using that for bulk files with only one record format.

My question is, is there a way to insert a conditional statement within the XML to handle different records within the same data file i.e.:

data1 data2 data 3

data4 data5 data6 data7 data8

etc etc…

I’ve looked through the schema for the XML and I’m coming to the conclusion that this is not what the bulk upload format file was designed for.

Thanks in advance!

Answer 1

To the best of my knowledge, there is no provided SQL Server import method (Bulk Insert, BCP, SSIS, etc.) that supports variable column sets. Every column defined absolutely must be in every import row.

For this reason, the preferred means of handling this (very common) situation is either of the following:

1) Use (or write) a specialized import program written in a general purpose language (VB, C#, etc.) to parse the text file and then insert it into your SQL Server’s table(s). Or,

2) Use one of SQL Server’s provided Import methods (Bulk Insert, BCP, SSIS, etc.) to import each row as a single text column (as NVarchar(MAX)) into a staging table and then use SQL to parse these text rows into their actual columns. (This is my preferred method)

A typical Staging Table for (2) above would look like this:

CREATE TABLE Import_Staging
(
    RowNumber INT IDENTITY(1,1) NOT NULL PRIMARY KEY,
    RowText   NVARCHAR(MAX)
);

If you do go this route, the import tools are generally happier if you hide the IDENTITY column from them, which can easily be done by creating a facade View and then directing the Import to the View instead of the Table. Like so:

CREATE VIEW vImport_Staging AS SELECT RowText FROM Import_Staging;