Question

I have a large dataset (~1GB) stored in a custom file format, the last 16 bytes of which is an MD5 hash of all previous bytes of the file.

I want to verify the MD5 of this file against the embedded MD5 using HashAlgorithm.ComputeHash(Stream), however this will calculate the hash of the entire file INCLUDING the hash in the last 16bytes, which obviously wont work.

How do I compute the MD5 hash of PART of a stream? I know I can read the stream into an array and pass this to HashAlgorithm.ComputeHash(Bytes), however the overhead of duplicating this 1GB of data in memory is prohibitive.

Answer 1

Taken from here where you can also get other ways of doing so.

Make a partial file stream class, read the size you want and make hash of it.

 class PartialFileStream : FileStream
{
    public PartialFileStream(string path, FileMode mode, long startPosition, long endPosition): base(path, mode)
{
  base.Seek(startPosition, SeekOrigin.Begin);
  ReadTillPosition = endPosition;
}

public long ReadTillPosition { get; set; }

public override int Read(byte[] array, int offset, int count)
{
 if (base.Position >= this.ReadTillPosition)
   return 0;

 if (base.Position + count > this.ReadTillPosition)
   count = (int)(this.ReadTillPosition - base.Position);

 return base.Read(array, offset, count);
  }
}