Question

I’m trying to append some data to a stream. This works well with FileStream, but not for MemoryStream due to the fixed buffer size.

The method which writes data to the stream is separated from the method which creates the stream (I’ve simplified it greatly in the below example). The method which creates the stream is unaware of the length of data to be written to the stream.

public void Foo(){
    byte[] existingData = System.Text.Encoding.UTF8.GetBytes("foo");
    Stream s1 = new FileStream("someFile.txt", FileMode.Append, FileAccess.Write, FileShare.Read);
    s1.Write(existingData, 0, existingData.Length);


    Stream s2 = new MemoryStream(existingData, 0, existingData.Length, true);
    s2.Seek(0, SeekOrigin.End); //move to end of the stream for appending

    WriteUnknownDataToStream(s1);
    WriteUnknownDataToStream(s2); // NotSupportedException is thrown as the MemoryStream is not expandable
}

public static void WriteUnknownDataToStream(Stream s)
{
   // this is some example data for this SO query - the real data is generated elsewhere and is of a variable, and often large, size.
   byte[] newBytesToWrite = System.Text.Encoding.UTF8.GetBytes("bar"); // the length of this is not known before the stream is created.
   s.Write(newBytesToWrite, 0, newBytesToWrite.Length);
}

An idea I had was to send an expandable MemoryStream to the function, then append the returned data to the existing data.

public void ModifiedFoo()
{
   byte[] existingData = System.Text.Encoding.UTF8.GetBytes("foo");
   Stream s2 = new MemoryStream(); // expandable capacity memory stream

   WriteUnknownDataToStream(s2);

   // append the data which has been written into s2 to the existingData
   byte[] buffer = new byte[existingData.Length + s2.Length];
   Buffer.BlockCopy(existingData, 0, buffer, 0, existingData.Length);
   Stream merger = new MemoryStream(buffer, true);
   merger.Seek(existingData.Length, SeekOrigin.Begin);
   s2.CopyTo(merger);
}

Any better (more efficient) solutions?

Answer 1

A possible solution is not to limit the capacity of the MemoryStream in the first place.
If you do not know in advance the total number of bytes you will need to write, create a MemoryStream with unspecified capacity and use it for both writes.

byte[] existingData = System.Text.Encoding.UTF8.GetBytes("foo");
MemoryStream ms = new MemoryStream();
ms.Write(existingData, 0, existingData.Length); 
WriteUnknownData(ms);

This will no doubt be less performant than initializing a MemoryStream from a byte[], but if you need to continue writing to the stream I believe it is your only option.