Question

I have a VB.net application with an Access Database with one table that contains about 2,800,000 records, each raw is updated with new data daily. The machine has 64GB of ram and i7 3960x and its over clocked to 4.9GHz.

Note: data sources are local.

I wonder if I use ~10 threads will it finish updating the data to the rows faster.

If it is possiable what would be the mechanisim of deviding this big loop to multiple threads?

Update: Sometimes the loop has to repeat the calculation for some row depending on results also the loop have exacly 63 conditions and its 242 lines of code.

Answer 1

Microsoft Access is not particularly good at handling many concurrent updates, compared to other database platforms.

The more your tasks need to do calculations, the more you will typically benefit from concurrency / threading. If you spin up 10 threads that do little more than send update commands to Access, it is unlikely to be much faster than it is with just one thread.

If you have to do any significant calculations between reading and writing data, threads may show a performance improvement.

I would suggest trying the following and measuring the result:

• One thread to read data from Access
• One thread to perform whatever calculations are needed on the data you read
• One thread to update Access

You can implement this using a Producer / Consumer pattern, which is pretty easy to do with a BlockingCollection.

The nice thing about the Producer / Consumer pattern is that you can add more producer and/or consumer threads with minimal code changes to find the sweet spot.

Supplemental Thought

IO is probably the bottleneck of your application. Consider placing the Access file on faster storage if you can (SSD, RAID, or even a RAM disk).