Question

i would like to insert records in set of batches. For example, lets say I have 10,000 records to be inserted. I want to insert them in batches of 500 or 1,000. How can I do this dynamically? That is…take the first 500 or 1,000 insert them, take the next and insert them, and so forth. In my example, the set of 10,000 is made up of a Generic List, so I want to insert, or divide up, this List accordingly.

The count of 10,000 will not be a static count. May be less or more. Just want to insert in batches of 500 or 1,000.

Answer 1

Using Linq-to-Sql to perform the INSERT‘s will not be efficient because it will always run them as individual INSERT statements even if you send in batches. You can try using a library or 3rd party code like this one for sending the batches: http://devio.wordpress.com/2011/08/05/batch-insert-using-linq-to-sql/.

As far as splitting up your lists, you can do something like this:

var first = myList.Take(500);
var second = myList.Skip(500).Take(500);
var third = myList.Skip(1000).Take(500);
// etc.

Obviously this could be done in a loop by incrementing the Skip amount instead of creating a separate variable for each section – I’m just showing the idea.

Also you may find these links helpful:

• Why massive inserts using SubmitChanges lack in performance.
• Batch Updates and Deletes with LINQ to SQL (I know this isn’t about INSERTing, but it gives some good background info on why Linq-to-Sql is inefficient at batching)