I’ve read many answers here about this topic, but everyone suggests the BCP || SqlBulkCopy class from .net
I have a query which inserts into targetTable the union of 5 selects from different tables.
I have correct indexes on the tables being selected. And only 1 clustered identity index on the targetTable. However this takes a long time (~25 min). I’m talking about 5M rows (x 20 columns).
When I look at sp_who2, most of the time, it is suspended…
I want to use bulk copy but not from .net (the db already fetches the data – so I don’t need to go to C#).
Questions
- How can I use bulk insert (no bcp) in my select command?
- Also, why is it suspended most of time? How can I give my query a higher priority?
Thank you.
p.s. I can’t use bcp here because of security restrictions… I don’t have permission to run this.
You’re right: This is taking longer than usual. You’re getting 3k rows per second. You should get 10k or 20k per second easily. In the best case 200k per second per CPU core
I suspect you are inserting all over the table, not just at the end. In this case, 3k rows per second is not unusual.
In any case, bulk copy cannot help you. It does not insert faster than a server-only insert statement.
What you can do, though, is insert using multiple threads. Partition your row source into N distinct ranges and insert each range concurrently from a separate connection. This will help if you are CPU bound. It won’t if you are IO bound.