Question

Earlier I made an HttpWebRequest that worked perfectly fine, and my StreamReader read the HTML of the website perfectly.

But all of the sudden, after having tested it’s functionality and confirmed that it worked many times, it hangs the program when it comes to the StreamReader line.
I have tried removing this line, and the code continued.

The thing is; I tried inputting a different website than the one I need to use, (I put in http://www.google.com) and it worked perfectly fine. So my error conclusion is, that it is only the website I need to use that I can’t access anymore which makes me think that the endpart (the website) is cancelling my connection or blocking me or something. BUT! The HttpWebRequest itself doesn’t hang or anything, which must mean that it successfully established a request to the website?

Enough chit-chat, here’s the code:

HttpWebRequest request = (HttpWebRequest)WebRequest.Create("website here");
MessageBox.Show("1"); //This is shown.
string HTMLLink = (new     StreamReader(request.GetResponse().GetResponseStream())).ReadToEnd(); //This is where the     program hangs....
MessageBox.Show("2"); //This is not shown! Below this isn't being executed.
if (HTMLLink.Length > 0)
{
    HTMLLink = HTMLLink.Substring(HTMLLink.IndexOf("uuu"), HTMLLink.Length -     HTMLLink.IndexOf("uuu"));
    HTMLLink = HTMLLink.Substring(0, HTMLLink.IndexOf("\" TARGET="));
    request = (HttpWebRequest)WebRequest.Create(HTMLLink);
    string HTML = (new     StreamReader(request.GetResponse().GetResponseStream())).ReadToEnd();
    if (HTML.Length > 0 && HTML.Contains(" </script><br><br><br>") && HTML.Contains("    <br><br><script "))
    {
        HTML = HTML.Substring(HTML.IndexOf(" </script><br><br><br>") + 22,     HTML.IndexOf("<br><br><script "));
        HTML = HTML.Substring(0, HTML.IndexOf("<br><br><script "));
        HTML = HTML.Replace("\r\n", "");
        HTML = HTML.Replace("\n", "");
        HTML = HTML.Replace("<br>", "\r\n");
        HTML = HTML.Replace("<BR>", "\r\n");
        HTML = HTML.Replace("<br />", "\r\n");
        HTML = HTML.Replace("<BR />", "\r\n");
        textBox.Text = HTML;
    }
}

And please keep in mind that it worked perfectly earlier then all of the sudden it started hanging, and that it works fine with http://www.google.com.

And by the way, yes I have done many searches. No useful results.

I have tried the timeout already, it does timeout.
Maybe the website has blocked my program thinking it’s a spider? what then?

Everytime when I reach the StreamReader (no matter how I set it up) it starts to hang.
And it keeps hanging, it doesn’t deliver any result.
This ONLY happens with lyrics007.com which is the exact website I need. It works fine with google.

Help, please!

Thanks in advance!

Answer 1

WebRequest.GetResponse() is a blocking call. It will wait until it can successfully connect and receive the response before it returns control to the caller, or will throw an exception if unsuccessful. This behaviour can’t be modified.

You usually don’t want your application to sit waiting for something to happen though, so you usually delegate the GetResponse() call to another thread, so you can continue doing other work in the current thread.

The usual way to overcome this problem is to call asynchronously. Rather than a call to GetResponse, you will call BeginGetResponse(), passing in a function which should be executed when the operation completes (eg, containing the remainder of your current method, plus a call to EndGetResponse()). Control of execution can be passed back to the caller whilst the response is being waited for in a background thread, handled for you automatically by the .NET threadpool.