Question

So I’m using a StreamReader that uses a MemoryStream to write to a StreamWriter and inside of this application but the memory usage increases by 300mb (From one of the larger inputs) and does not deallocate after Im done using it:

StreamWriter log = new StreamWriter("tempFile.txt");
log.Write(reader.ReadToEnd());
log.Close();

reader.DiscardBufferedData();
reader.Close();
reader.Dispose();
memoryStream.Dispose();
log.Dispose();
GC.Collect();

Before this and right after I get the RAM usage and before it is 300 mb less than after but I don’t know why. I have done everything I can think of to release that memory considering that the only thing going on here is the data from the reader is being placed in the text file I don’t see why any large amount of memory would even need to be used temporarily. Is there something I am missing?… Thanks.

Answer 1

Are you looking at the RAM used by the process itself? I wouldn’t expect that to go down. The process will hang on to that memory and reuse it for further allocations. It doesn’t hand it back to the operating system. That’s just the way .NET tends to work.

You can potentially get it to throw away the memory with some CLR API call – but usually I wouldn’t.

This doesn’t mean the memory has really leaked – it’s still usable by the same process. For example, if you performed the same action again it probably wouldn’t need to increase the size of the heap – you’d see the memory usage stay flat at the process level. Use the CLR perfmon graphs to see the memory used within the managed heap to see if there’s a genuine leak.

(There are also various different measure of how much memory an application is actually using. Which one is interesting depends on what you’re trying to do.)

EDIT: If your code snippet is genuinely indicative of the code you’re using, then there’s a much simpler alternative: stream the data.

using (TextWriter writer = File.CreateText("tempFile.txt"))
{
    CopyText(reader, writer);
}

static void CopyText(TextReader reader, TextWriter writer)
{
    char[] buffer = new char[8192];
    int charsRead;
    while ((charsRead = reader.Read(buffer, 0, buffer.Length)) > 0)
    {
        writer.Write(buffer, 0, charsRead);
    }
}

Note that unless you’re actually changing the encoding, you could do this without using a TextWriter/TextReader pair to start with.

That way you won’t need to have the whole string in memory as well as its binary representation (in the MemoryStream). You’ll still have the binary representation of course – do you have to write everything into the MemoryStream to start with?