Question

I’m trying to read the reponse from a webserver using httpwebrequests in C#.
I use the following code:

UriBuilder urib = new UriBuilder();
urib.Host = "wikipedia.com";

HttpWebRequest req = WebRequest.CreateHttp(urib.Uri);
req.KeepAlive = false;
req.Host = "wikipedia.com/";
req.Method = "GET";

HttpWebResponse response = (HttpWebResponse) req.GetResponse();
byte[] buffer = new byte[response.ContentLength];
System.IO.Stream stream = response.GetResponseStream();
stream.Read(buffer, 0, buffer.Length);

Console.WriteLine(System.Text.Encoding.ASCII.GetString(buffer, 0, buffer.Length));

The code does indeed retrieve the correct amount of data (I compared the contentlength used to create the buffer, with the length of the console output, they’re the same.
My problem is that the last 80% or so of the response is blank chars. They’re all 0x00.
I tested this with several pages, including wikipedia.com and it just cuts off mid-file for some reason.

Have I misunderstood/misused the way to use webrequests or can anyone spot an error here?

Answer 1

Try to use this method:

public static String GetResponseString(Uri url, CookieContainer cc)
{
    HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
    request.Method = WebRequestMethods.Http.Get;
    request.CookieContainer = cc;
    request.AutomaticDecompression = DecompressionMethods.GZip;

    HttpWebResponse response = (HttpWebResponse)request.GetResponse();

    StreamReader reader = new StreamReader(response.GetResponseStream());

    String responseString = reader.ReadToEnd();

    response.Close();

    return responseString;
}