Question

this is in references to my previous question Log to memory and then write to file, actually the edit part of that question, I asked in edit part that if I write to memory would that be faster than writing to file? I performed a simple test, and I had shocking results! I wanted to share with the community.
So here’s the code

private void Button1Click(object sender, EventArgs e)
{
    var stopwatch = new Stopwatch();
    stopwatch.Start();
    File.AppendAllText(@"D:\File1.txt", string.Format("{0}Start! : {1}", Environment.NewLine, DateTime.Now.ToString(CultureInfo.InvariantCulture)));
    for (int i = 0; i < 6; i++)
    {
        for (int j = 0; j < 1000000; j++)
        {
             File.AppendAllText(@"D:\File1.txt", string.Format("{0}{1}:{2}", Environment.NewLine, i.ToString(CultureInfo.InvariantCulture), j.ToString(CultureInfo.InvariantCulture)));
         }
    }
    File.AppendAllText(@"D:\File1.txt", string.Format("{0}Done!{1}", Environment.NewLine, DateTime.Now.ToString(CultureInfo.InvariantCulture)));
    stopwatch.Stop();
    File.AppendAllText(@"D:\File1.txt",
                       string.Format("{0}{1}:{2}",Environment.NewLine,              stopwatch.Elapsed.ToString(), stopwatch.ElapsedMilliseconds.ToString(CultureInfo.InvariantCulture)));
    MessageBox.Show("Done!");
}

private void Button2Click(object sender, EventArgs e)
{
     var stopwatch = new Stopwatch();
     using (var mem = new MemoryStream())
     {
         using (var binaryWriter = new BinaryWriter(mem))
         {
             stopwatch.Start();
             {
                 binaryWriter.Write("start! : " + DateTime.Now.ToString(CultureInfo.InvariantCulture));
                 for (int i = 0; i < 6; i++)
                 {
                     for (int j = 0; j < 1000000; j++)
                     {
                         binaryWriter.Write(i.ToString(CultureInfo.InvariantCulture) + ":" + j.ToString(CultureInfo.InvariantCulture));
                     }
                 }
                 stopwatch.Stop();
                 binaryWriter.Write("Done! " + DateTime.Now.ToString(CultureInfo.InvariantCulture));
                 binaryWriter.Write(stopwatch.Elapsed.ToString() + ":" + stopwatch.ElapsedMilliseconds.ToString(CultureInfo.InvariantCulture));
                 binaryWriter.Flush();
                 var file = new FileStream(@"D:\File2.txt", FileMode.Create);
                 mem.WriteTo(file);
             }
         }
     }
     MessageBox.Show("Done!");
}

As the code should be easy to understand

Elapsed time in File1.txt = 00:50:24.5654918  
Elapsed milliseconds in File1.txt = 3024565  
Elapsed time in File2.txt = 00:00:04.7430152  
Elapsed milliseconds in File2.txt = 4743

So, as you can see for yourself, there is about 50 minutes of differences! This could be a real cause for bad perfromance, if you log everything directly to IO File, without use of memory stream or any custom tool for logging, OTOH compared to 50 minutes of File.AppendAllText, using MemoryStream only took about 4 and a quarter second. (I am still confused as to why the time shown in windows explorer doesn’t corresponds to time shown in file in the end by stopwatch.ShowElapasedTime, but nonetheless, even if we see windows explorer time, its still about 45 minutes faster!) So, this can be a really useful thing, I thought of sharing it!

Answer 1

That’s because File.AppendAllText opens the file, writes, flushes the buffer and closes it. If you keep the log file open and use a stream to write to it (instead of the MemoryStream), you will get results that are very close to what you’ve seen with MemoryStream – it might even be indistinguishable.

Try it out.