Question

I have a 3rd party app that does a database export in to a “SQL format” it is fairly straitforward however it appears to not use T-SQL exactly. When I export it produce something like this

INSERT INTO [ExampleDB] ( [IntField] , [DateTimeField] , [VarcharField], [BinaryField])
VALUES
(1 , '2012/04/02 12:25:00:01' , 'Some Text' , X'123456'),
(2 , '0000/00/00 00:00:00:00' , 'B' , NULL),
--(SNIP, it does this for 1000 records)
(999,'0000/00/00 00:00:00:00' , 'Other Text' , null);
(1000 ,'0000/00/00 00:00:00:00' , 'D' , null);

INSERT INTO [ExampleDB] ( [IntField] , [DateTimeField] , [VarcharField] , BinaryField)
VALUES
(1001 , '2012/04/02 12:25:00:01' , 'Trying to break my parser with the next line', null),
(1002 , '0000/00/00 00:00:00:00' , ' ''X''123' , X'deadbeef'),
(1003 , '0000/00/00 00:00:00:00' , 'Did it break it?' , null),
(1004 , '2012/04/02 12:25:00:01' , 'What about this? ''0000/00/00 00:00:00:00'' Will that?' , null),
--(SNIP)

The two snags I am running in to are:

• the value of '0000/00/00 00:00:00:00' for a date
• the fact that they store binary output as a string prefixed by X

I am thinking I can just do a Regex.Replace on it, but I am wondering if there is a easier solution. So far my code to import is

using (var cmd = new SqlCommand("", conn))
{
    //snip
    cmd.CommandTimeout = 0; //Wait forever, bad bad bad, but what else can I do?
    using (var txtRdr = new StreamReader(file))
    {
        string query = txtRdr.ReadToEnd();
        query = query.Replace(" '0000/00/00 00:00:00:00' ", " NULL ");
        query = Regex.Replace(query, "X'([0-9a-fA-F]+)'", @"0x$1");
        cmd.CommandText = query;
        cmd.ExecuteNonQuery();

    }
}

Because the way they list values I know I am tied to SQL2008 or newer but are there any gotchas to the code I wrote to make it parse-able by SQL2008?

When I run it it gets bogged down on the Execute query. The SQL files are 8000 KB in size (but I can adjust it) is there anything I can do to speed it up? Perhaps do some more formatting and turn it in to a bulk insert?

Answer 1

Whenever I have had to transfer data between two databases I have found that the quickest way seems to be as follows:

1. Use the unload or equivalent feature to bulk export the source database to a text file.
2. Write software that converts the text file format from the source format to the destination format.
3. Use the bulk import or similar feature to load the text file into the destination database.