Question

This is just an imaginary problem, I’m hoping that the solution will help in whole range of similar scenarios. Suppose I need to count total size of all external resources on a webpage (images, scripts etc.). I download the page, extract all SRC information and transform the URL list into download tasks:

async Task<int> GetTotalSize(Uri uri) {
  string[] urls = ... code to extract all external resources' URLs from given page ...

  var tasks = from url in urls.Distinct()
                select new WebClient().DownloadDataTaskAsync(new Uri(url));
  var files = await TaskEx.WhenAll(tasks);
  return files.Sum(file => file.Length);
}

Now, if one of the links is unreachable for any reason, the whole TaskEx.WhenAll is aborted with WebException. What I need is to ignore any WebExceptions inside individual tasks and assume length of 0 in that case. Any ideas?

Answer 1

Just add a separate (asynchronous) method to get the size of a single url. Then add them up.

e.g.,

static async Task<int> GetTotalSizeAsync(params string[] urls)
{
    if (urls == null)
        return 0;
    var tasks = urls.Select(GetSizeAsync);
    var sizes = await TaskEx.WhenAll(tasks);
    return sizes.Sum();
}

static async Task<int> GetSizeAsync(string url)
{
    try
    {
        var str = await new WebClient().DownloadStringTaskAsync(url);
        return str.Length;
    }
    catch (WebException)
    {
        return 0;
    }
}