Question

I am working on a process at the moment which iterates through files in a folder; its used as part of a data migration. Each iteration of a file takes about 8 seconds; I need to run it about 30000 times so 8s that is a bit of a problem. The actions in the function cant be multithreaded as it pulls apart the document and that needs to be serial.

The single threaded code does this

    For Each strFile In System.IO.Directory.GetFiles(txtFolderPathIN.Text)
        CallFunction(strFile)
    Next

What is the best approach to convert this to make it multithreaded? There is no feedback to the user; just need to start the process and iterate through them as quickly as possible. What the easiest way to making this multithreaded?

Answer 1

I would use a ThreadPool. Something like this:

ThreadPool.SetMaxThreads = 4

For Each strFile In System.IO.Directory.GetFiles(txtFolderPathIN.Text)
    ThreadPool.QueueUserWorkItem(new WaitCallback(addressof CallFunction), strFile)
Next

You could also use your own thread pooling system by having a list of threads and using a mutex to stop the main thread from exiting before all the child threads have finished running.