Question

When it comes down to INSERTing large amount of data (say 100.000 rows) in an ARRAY object,
which one do you think it would be faster?

1. Through Entity Framework? (call SaveChanges() when u have inserted 100.000 rows)
2. Looping every row (100.000 times) with an INSERT stored procedure?

If you could also provide a reference that would be very awesome.

Thanks

Answer 1

Looping 100K times calling a stored procedure will at a minimum create 100K cross process and/or cross network calls will be slow.

If you’re using SQL server, another option is to use TVPs (table value paramaters) to avoid calling insert in a loop from your C# code. It allows you to pass a table of data to a stored procedure in one call.

From the link above, they recommend 1000 rows at a time (but always measure and experiment for your app):

Using table-valued parameters is comparable to other ways of using
set-based variables; however, using table-valued parameters frequently
can be faster for large data sets. Compared to bulk operations that
have a greater startup cost than table-valued parameters, table-valued
parameters perform well for inserting less than 1000 rows.

So, maybe try out looping 100 times passing a 1000 rows at a time (instead of cross the boundary 100K times).

You might also want to re-evaluate why asp.net has 100K items at one time in your app. Is that passed up to the server and held in memory at once with possible memory issues? Can that be broken up? Are you doing data processing where asp.net is reading out and processing 100K rows where a sql server agent job might be more appropriate? If you provide more details on the data flow of your app and what it’s doing, folks might be able to offer more options.