Question

I’m working with large files , beginning from 10Gb. I’m loading the parts of the file in the memory for processing. Following code works fine for smaller files (700Mb)

 byte[] byteArr = new byte[layerPixelCount];
 using (FileStream fs = File.OpenRead(recFileName))
    {
        using (BinaryReader br = new BinaryReader(fs))
        {
            fs.Seek(offset, SeekOrigin.Begin);

            for (int i = 0; i < byteArr.Length; i++)
            {
                byteArr[i] = (byte)(br.ReadUInt16() / 256);
            }
         }
    }

After opening a 10Gb file, the first run of this function is OK. But the second Seek() throws an IO exception:

An attempt was made to move the file pointer before the beginning of the file.

The numbers are:

fs.Length = 11998628352

offset = 4252580352

byteArr.Length = 7746048

I assumed that GC didn’t collect the closed fs reference before the second call and tried

    GC.Collect();
    GC.WaitForPendingFinalizers();

but no luck.

Any help is apreciated

Answer 1

I’m guessing it’s because either your signed integer indexer or offset is rolling over to negative values. Try declaring offset and i as long.

//Offest is now long
long offset = 4252580352;

byte[] byteArr = new byte[layerPixelCount];
using (FileStream fs = File.OpenRead(recFileName))
{
   using (BinaryReader br = new BinaryReader(fs))
    {
        fs.Seek(offset, SeekOrigin.Begin);

        for (long i = 0; i < byteArr.Length; i++)
        {
            byteArr[i] = (byte)(br.ReadUInt16() / 256);
        }
    }
}