Question

I had 1 collection which I was saving the content to a persistent storage. Then I ended up having 3 collections which i like to save to a persistent storage.

At first I was using a ThreadPool to save one collection, but now I have 3 collections.Since each collection goes to a different storage, I dont want to combine them or save to same place.

My question is should i use manual Threads and create one thread for each Save() method or should I create 3 thread pool for each method, or should i call all the 3 methods in one ThreadPool.QueueUserWorkItem call.

1.First Approach

ThreadPool.QueueUserWorkItem(o => 
             { Save<Foo>(ConcurrentCollectionStorage.Bus1);
               Save<Bar>(ConcurrentCollectionStorage.Bus2);
               Save<Car>(ConcurrentCollectionStorage.Bus3);
             });

2. Second Approach

ThreadPool.QueueUserWorkItem(o =>
               { Save<Foo>ConcurrentCollectionStorage.Bus); });  
ThreadPool.QueueUserWorkItem(o =>
               { Save<Bar>(ConcurrentCollectionStorage.Bus2); });  
ThreadPool.QueueUserWorkItem(o =>
               { Save<Car>(ConcurrentCollectionStorage.Bus3); });  

3. Third Approach. Creating Thread Manually and join them.

While doing these operations I dont want my application to hang. I want it to process and save the data, and complete, but not to affect the foreground processes, the whole application.

What s the best way to do this?

Which one should I use? Is there a better way to do this?

Answer 1

With .NET 4.0 you can use the Task Parallel Library:

    Task.Factory.StartNew(() => { Save<Foo>(ConcurrentCollectionStorage.Bus1); });
    Task.Factory.StartNew(() => { Save<Bar>(ConcurrentCollectionStorage.Bus2); });
    Task.Factory.StartNew(() => { Save<Car>(ConcurrentCollectionStorage.Bus3); });

However, it isn’t clear from your question what the critical bottleneck is. The TPL can’t help you if you’re bottleneck is Disk IO or network latency.

If you want your main thread to wait for them all to complete, you can do this (there are several ways to do it):

    Task t1 = Task.Factory.StartNew(() => { Save<Foo>(ConcurrentCollectionStorage.Bus1); });
    Task t2 = Task.Factory.StartNew(() => { Save<Bar>(ConcurrentCollectionStorage.Bus2); });
    Task t3 = Task.Factory.StartNew(() => { Save<Car>(ConcurrentCollectionStorage.Bus3); });

    Task.WaitAll(t1,t2,t3);

This way the thread running the tasks will wait until they finish. t1,t2, and t3 will be running asynchronously.