So I have this one time deal where I have to get data from different databases/tables, slice and dice it and insert it into one single table.
This is done through a Windows Form C# .NET 4.0 application.
I have to run this one time deal process once (obviously) for each different entities we’ll be using in our CRM environment.
For some entities, only 20,000 rows to insert so a networked SQL connection is just fine.
But this other one is 800,000 long and is taking quite a long time to finish.
I was wondering whether I’d gain any speed by installing the application on the SQL Server itself and specifying a local connection in my connection string instead of a networked one.
I’d have to make formal request to gain remote desktop access to that server, which is why I’m asking if anyone else has ever used that approach successfully here before actually testing it.
Also, I’m doing my inserts through Entity Framework 4’s SaveChanges() which I call at each inserted row. Perhaps, there is room for optimization there as well and I’m willing to read your opinions.
Thanks much!
Anytime you can eliminate the network you will see a performance improvement. For 800k rows, I would use the SqlBulkCopy class. It’s very easy to use, it basically does the equivalent of a BULK INSERT under the covers.