Question

Thread-Safety is not an aspect that I have worried about much as the simple apps and libraries I have written usually only run on the main thread, or do not directly modified properties or fields in any classes that I needed to worry about before.

However, I have started working on a personal project that I am using a WebClient to download data asynchronously from a remote server. There is a Queue<Uri> that contains a pre-built queue of a series of URI’s to download data.

So consider the following snippet (this is not my real code, but something I am hoping illustrates my question:

private WebClient webClient = new WebClient();
private Queue<Uri> requestQueue = new Queue<Uri>();

public Boolean DownloadNextASync()
{
    if (webClient.IsBusy)
        return false;

    if (requestQueue.Count == 0)
        return false

    var uri = requestQueue.Dequeue();

    webClient.DownloadDataASync(uri);

    return true;

}

If I am understanding correctly, this method is not thread safe (assuming this specific instance of this object is known to multiple threads). My reasoning is webClient could become busy during the time between the IsBusy check and the DownloadDataASync() method call. And also, requestQueue could become empty between the Count check and when the next item is dequeued.

My question is what is the best way to handle this type of situation to make it thread-safe?

This is more of an abstract question as I realize for this specific method that there would have to be an exceptionally inconvenient timing for this to actually cause a problem, and to cover that case I could just wrap the method in an appropriate try-catch since both pieces would throw an exception. But is there another option? Would a lock statement be applicable here?

Answer 1

If you’re targeting .Net 4.0, you could use the Task Parallel Library for help:

var queue = new BlockingCollection<Uri>();
var maxClients = 4;

// Optionally provide another producer/consumer collection for the data
// var data = new BlockingCollection<Tuple<Uri,byte[]>>();

// Optionally implement CancellationTokenSource

var clients = from id in Enumerable.Range(0, maxClients)
              select Task.Factory.StartNew(
    () =>
    {
        var client = new WebClient();
        while (!queue.IsCompleted)
        {
            Uri uri;
            if (queue.TryTake(out uri))
            {
                byte[] datum = client.DownloadData(uri); // already "async"
                // Optionally pass datum along to the other collection
                // or work on it here
            }
            else Thread.SpinWait(100);
        }
    });

// Add URI's to search
// queue.Add(...);

// Notify our clients that we've added all the URI's
queue.CompleteAdding();

// Wait for all of our clients to finish
clients.WaitAll();

---

To use this approach for progress indication you can use TaskCompletionSource<TResult> to manage the Event based parallelism:

public static Task<byte[]> DownloadAsync(Uri uri, Action<double> progress)
{
    var source = new TaskCompletionSource<byte[]>();
    Task.Factory.StartNew(
        () =>
        {
            var client = new WebClient();
            client.DownloadProgressChanged
                += (sender, e) => progress(e.ProgressPercentage);
            client.DownloadDataCompleted
                += (sender, e) =>
                {
                    if (!e.Cancelled)
                    {
                        if (e.Error == null)
                        {
                            source.SetResult((byte[])e.Result);
                        }
                        else
                        {
                            source.SetException(e.Error);
                        }
                    }
                    else
                    {
                        source.SetCanceled();
                    }
               };
        });

    return source.Task;
}

Used like so:

// var urls = new List<Uri>(...);
// var progressBar = new ProgressBar();

Task.Factory.StartNew(
    () =>
    {
       foreach (var uri in urls)
       {
           var task = DownloadAsync(
               uri,
               p =>
                   progressBar.Invoke(
                       new MethodInvoker(
                       delegate { progressBar.Value = (int)(100 * p); }))
               );

           // Will Block!
           // data = task.Result;
       } 
    });