Question

I have created a new WebClient class that has a longer Request.Timeout property by inheriting the native WebClient class and overriding the GetWebRequest method:

public class LongerTimeoutWebClient : WebClient
{
    public LongerTimeoutWebClient()
    {
    }

    protected override WebRequest GetWebRequest(Uri address)
    {
        WebRequest r = base.GetWebRequest(address);
        r.Timeout = 30 * 1000; //30 second timeout
        return r;
    }
}

I use this new WebClient class to call a number of web pages in Parallel:

Parallel.ForEach(links, new ParallelOptions { MaxDegreeOfParallelism = 2 }, link =>
        {
            byte[] requestBytes;

            var stopwatch = new System.Diagnostics.Stopwatch();
            using (var client = new LongerTimeoutWebClient())
            {
                try
                {
                    ServicePointManager.Expect100Continue = false;

                    stopwatch.Start();
                    requestBytes = client.DownloadData(link);
                }
                catch (Exception ex)
                {
                    stopwatch.Stop();
                    Console.WriteLine("Could not get: " + link + ". Timeout after: " + stopwatch.Elapsed.TotalSeconds + "s");
                    Console.WriteLine(ex);
                    return;
                }
            }
//More code here
}

However, I am intermittently getting a 408 Request Timeout error. The stopwatch value when Timeout error occurs is around 20s.

I have also checked the Timeout property of the Request object used by the GetWebResponse method of the new WebClient class and it is set to 30000 (ms).

I have set the MaxDegreeOfParallelism to 1 and I still get the errors, so that probably means the parallelization is not the cause of the problem.

Why is my Request getting timed out at 20s instead of 30s?

Answer 1

There are two kind of timeouts. Client timeout and server timeout. Probably in this case what you are encountering is server timeout. Check it once.