Question

In the past I have used ThreadPool.QueueUserWorkItem to spawn multiple threads from a manager class. This manager class subscribes to an event in these spawned threads which is raised when the thread work completes. The manager class can then handle writing output to a text file by using a lock to prevent any race conditions.

Now I am using Parallel.ForEach to do the work. What is my best method for writing all output to a text file in a thread safe manner?

The basic outline of my implementation:

public class Directory
{
    public string Path;

    public Directory(string path)
    {
        Path = path;
    }

    public void Scan()
    {
        Parallel.ForEach(new DirectoryInfo(Path).GetDirectories(),
                         delegate(DirectoryInfo di)
                         {
                             var d = new Directory(di.FullName);
                             d.Scan();
                             //Output to text file.
                         });

    }
}

Which I get going by:

new Directory(@"c:\blah").Scan();

Any ideas to point me in the right direction would be great. I have a few myself but I am looking for best practice. I have read this post but it does not contain any solution that helps me.

Answer 1

Use EnumerateDirectories (Fx 4) instead of GetDirectories. Your current code would not work in parallel very much.

For the rest it depends on whether you need the output to be in order or not.
If you don’t care about the order, you can simply lock the output stream (with a helper object), write and continue. No need for a complicating Event.
If you want to maintain order,

Push the output to a Queue. Process the queue when the ForEach is complete or start a separate Task (Consumer) to write it ASAP. This would be a typical Producer/Consumer pattern.

Please note that by making the processing Parallel it becomes very difficult to maintain the order in which the Directories are written.