Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
I want transfer data from MongoDB to SQL Sever using the C# driver.
The data size is about 20,000,000 in three collections. I’m using 3 thread to
read it and 3 threads to insert into 3 different SQL Server tables.
After reading about 70,000,000 documents, the performance becomes
worse, the speed about one tenth of before, and network
bandwidth slumps.
How can I prevent performance from dropping off?
Try migrating each collection sequentially on a single thread. The operation will be IO bound rather than CPU bound, so running it multi-threaded will more likely degrade performance and increase resource contention and context switching in the databases.
Also take a look at your SQL Server database disk writes. It might be that the hard disk cache is being used but then writes slow down as the cache is flushed to disk.