Question

I am trying to read 500 MB text file to send it contents through HttpWebRequest. According to my requirement, I cannot send the data in chunks. Code is as follows :

using (StreamReader reader = new StreamReader(filename))
{
    postData = reader.ReadToEnd();
}

byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "text/plain";
request.ContentLength = byteArray.Length;

Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();

WebResponse response = request.GetResponse();
Console.WriteLine(((HttpWebResponse)response).StatusDescription);

dataStream = response.GetResponseStream();

using (StreamReader reader = new StreamReader(dataStream))
{
    responseFromServer = reader.ReadToEnd();
}

Console.WriteLine(responseFromServer);
dataStream.Close();
response.Close();

Reading such large file gives me out of memory exception. Is there a way I can do this?

Answer 1

Sounds like you may be encountering this documented issue with HttpWebRequest. Per the KB article, try setting the HttpWebRequest.AllowWriteStreamBuffering property to false.