Question

I’m trying to start async task (on .NET 4.5) which downloads content of web page, but somehow this task never finishes.

My PageDownloader class:

using System.Net;
using System.Text;
using System.IO;
using System.Net.Http;
using System.Threading.Tasks;
using System;

namespace ParserConsole.WebClient
{
public class PageDownloader
{
    private System.Net.Http.HttpClient _client;

    public PageDownloader()
        : this(Encoding.UTF8) { }

    private Encoding _encoding;

    public PageDownloader(Encoding encoding)
    {
        _encoding = encoding;
        _client = new HttpClient() { Timeout = TimeSpan.FromSeconds(10)};
    }

    private HttpRequestMessage _request;
    private HttpResponseMessage _response;
    private string _responseString;

    public string GetPageData(string link)
    {
        _request = new HttpRequestMessage(HttpMethod.Get, link);
        _request.Headers.Add("User-Agent", "Chrome/21.0.1180.89");
        _request.Headers.Add("Accept", "text/html");


        GetResponse().Wait();
        GetStringFromResponse().Wait();
        return _responseString;            
    }

    private async Task<HttpResponseMessage> GetResponse() {
        return _response = await _client.GetAsync(_request.RequestUri);
    }

    private async Task<string> GetStringFromResponse() {
        return _responseString = await _response.Content.ReadAsStringAsync();
    }

}
}

I start downloading page by calling

new PageDownloader().GetPageData(url);

When I’m trying to debug the code, everything is fine till GetResponse().Wait(). But somehow GetResponse() task never finishes – breakpoint on the next line is never reached. I get no exceptions, application continues running. Any suggestions?

Answer 1

This is a standard deadlock condition you get when you start an async operation and then block on the returned task.

Here is a blog post discussion the topic.

Basically, the await call ensures that the continuation it wires up of the task will run in the context you were originally in (which is very helpful) but because you are calling Wait in that same context it’s blocking, so the continuation never runs, and that continuation needs to run for the wait to end. Classic deadlock.

As for the fix; usually it means you just shouldn’t be doing a blocking wait on the async operation; it’s contrary to the design of the whole system. You should, “async all the way up”. In this case it would mean that GetPageData should return a Task<string> rather than a string, and rather than waiting on the other operations that return a task you should await on them.

Now, having said that, there are ways of doing a blocking wait on the async operations without deadlocking. While it can be done, it honestly defeats the purpose of using async/await in the first place. The primary advantage of using that system is that the main context isn’t blocked; when you block on it that entire advantage goes away, and you might as well just use blocking code all the way through. async/await is really more of an all-or-nothing paradigm.

Here is how I would structure that class:

public class PageDownloader
{
    private System.Net.Http.HttpClient _client;
    private Encoding _encoding;

    public PageDownloader()
        : this(Encoding.UTF8) { }

    public PageDownloader(Encoding encoding)
    {
        _encoding = encoding;
        _client = new HttpClient() { Timeout = TimeSpan.FromSeconds(10) };
    }

    public async Task<string> GetPageData(string link)
    {
        HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, link);
        request.Headers.Add("User-Agent", "Chrome/21.0.1180.89");
        request.Headers.Add("Accept", "text/html");

        HttpResponseMessage response = await _client.GetAsync(request.RequestUri);

        return await response.Content.ReadAsStringAsync(); ;
    }
}