Question

I mean can I do something like this:

  var client = new WebClient(); 

  var result = client.DownloadString(string("http://example.com/add.php");

  var result2 = client.DownloadString(string("http://example.com/notadd.php"));

in paralel like for 100 url’s ?

Answer 1

In .NET 4.0, the simplest way is to use the ParallelExtensionsExtras‘s AsycCache along with the DownloadStringTask extension method. In fact, the example for this code covers your exact scenario:

public sealed class HtmlAsyncCache : AsyncCache<Uri, string>
{
    public HtmlAsyncCache() : 
        base(uri => new WebClient().DownloadStringTask(uri)) { }
}

...

HtmlAsyncCache cache = new HtmlAsyncCache();

var page1 = cache.GetValue(new Uri(“http://msdn.microsoft.com/pfxteam”));
var page2 = cache.GetValue(new Uri(“http://msdn.com/concurrency”));
var page3 = cache.GetValue(new Uri(“http://www.microsoft.com”)); 

Task.Factory.ContinueWhenAll(
    new [] { page1, page2, page3 }, completedPages =>
{
    … // use the downloaded pages here
});

See here for more details.