Question

I use multi threading to copy file to another place.I from stream needed byte array and dispose stream.at this example i use 7 threads to copy 3gb file.1st thread can get byte array,but at 2nd thread occurs exception ‘System.OutOfMemoryException’

public void Begin()
{
    FileStream stream = new FileStream(pathToFile, FileMode.Open);
    stream.Position = (threNmb - 1) * 536870912;
    BinaryReader reader = new BinaryReader(stream);
    for (long i = 0; i < (length); i++)
    {
        source.Add(reader.ReadByte());//gives exception at i=134217728
    }
    reader.Dispose();
    reader.Close();
    stream.Dispose();
    stream.Close();
}

Answer 1

It looks like you’re using a List<byte>. That’s going to be a very inefficient way of copying data, and you’re probably making it less efficient by using multiple threads. Additionally, if you’re using a single list from multiple threads, your code is already broken – List<T> isn’t thread-safe. Even if it were, you’d be mixing the data from the different threads, so you wouldn’t be able to reassemble the original data. Oh, and by not using using statements, if an exception is thrown you’re leaving file handles open. In other words, I’m advising you to completely abandon your current approach.

Instead, copying a file from one place to another (assuming you can’t use File.Copy for some reason), should basically be a case of:

• Open stream to read
• Open stream to write
• Allocate buffer (e.g. 32K)
• Repeatedly:
  • Read from the “input” stream into the buffer, noting how much data the call actually read
  • Write the data you’ve just read into the output stream
  • Loop until your “Read” indicates the end of the input stream
• Close both streams (with using statements)

There’s no need to have everything in memory. Note that in .NET 4 this is made even easier with the Stream.CopyTo method. which does the third and fourth steps for you.