I have a MS sql database (not server), my program reads file structure from my 2 TB external drive and saves the list a records in database. I write around 80k records every time and it takes a great time.
I am currently writing records by dividing them into 2 background Worker thread 40k each. My pc is Quad core Phenom II.
What is the best way to achieve this and also would there be any benefits by increasing the number of threads because mine is only a 7200rpm hdd and not raid. What I mean is HDD might be a limitation?
Thank you.
Edit: Time is write all records is around 15-18 mins.I am not using stored procedures. I am using the insert command from the auto generated table adapters only. I am looping the insert statement 40k times for each thread. No, not updating based on indexed file, they are just insert statements.
80k records shouldn’t take a ton of time. You say a “great time” but what is it actually? We have no idea of your relative term’s actual time.
1) are you using stored procedures?
2) Is it something that can be handled in bulk (BCP or with many insert/update statements generated and sent in a single command)?
3) Are all updates using indexed fields in the WHERE clause?