Question

Im experiencing problem with webclient to parse some very sensitive webpage. Somehow, webclient failed to do some task, and i dont even know why, no exception, no error, just simple not working (this is not the problem from the site)

I run webclient in a loop, however, only the first request was success, all the next requests are failed. When i restart the application, the same result appear, the first request always succeed, while all other request are failed.

Im sure all webclient are disposed properly, but i don’t know what is the problem, is webclient saving information even after dispose?

Please guide me how to clear everything, reset everything and start brand new webclient.

for (int i = 1; i <= Count; i++)
{
    using (WebClient wc = new WebClient())
    {
        wc.Headers["Accept-Encoding"] = "gzip";
        wc.Headers["User-Agent"] = "Mozilla/4.0 (Compatible; Windows NT 5.1; MSIE 6.0) (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)";
        byte[] arr = wc.DownloadData(url);

        if (arr.Length > 0)
            Console.WriteLine(i.ToString() + ": SUCCESS");
        else
            Console.WriteLine(i.ToString() + ": FAILED");
    }
}

Answer 1

WebClient will not share any information between instances.
You probably have a different problem.

For example, the site might ignore requests that happen too frequently.

You should run Fiddler and check exactly what is happening.

Also, try adding Thread.Sleep(5000) in the loop and see whether anything changes. (In Fiddler)