Question

We have a lengthy database operation. When we perform this operation with enlist = true, it takes 29 minutes to complete. When we perform this operation with enlist = false, it takes only 15 minutes to complete. What is the reason for such a huge performance hit? Is it something related to the database server?

How do we avoid it? Please share your thoughts.

Database: SQL Server 2005

Frameowrk: .Net 3.0

Answer 1

How do we avoid it?

and

Total number of rows in a csv file is 100000.

In this case, use SqlBulkCopy to write the 100,000 rows into a staging table (a basic and separate table, not part of your transactional tables; no triggers, foreign keys, etc). Once the data is in the staging table, use TSQL (either direct or a SPROC) to do the INSERT from the staging table into the transactional table, i.e. where your data lives. The transaction only needs to span this last operation.

This achieves several key things:

• SqlBulkCopy uses the same API as bulk-insert; it is the fastest way to shift large amounts of data over the wire to SQL Server
• it minimises the size of the transaction, most notably by not including all the data transfer time in the transaction; only the core INSERT is transactional, but at this point all the data is local