Question

Environment:
3 GB data file with a 28 byte SHA-1 checksum appended to the end.

Is it possible to use a FileStream or some other type of stream to calculate a SHA-1 checksum hash by reading the stream from position 0 to position [x – 28(The length of the appended checksum)] without physically removing the appended checksum from the end of the file first?

Currently we are opening the file using a FileStream and then reading it using a BinaryReader to seek and read the last 28 bytes of the file.

Then opening the file again with a FileStream and setting the length of the stream to the length – 28 bytes and this is removing the checksum from the file.

Finally we are calculating a checksum of the data file with the last 28 bytes removed and then comparing it to the checksum we removed from the end of the file to ensure the checksums match.

Basically I want to know if it’s possible to calculate the checksum of the data part of the file without having to remove the appended checksum first.

Answer 1

If you know how much data you really want to read (i.e. length – 28 bytes) then when you compute the hash, only put in that much data. You haven’t said exactly how you’re computing the hash, but you can generally repeatedly “write” more data into the hash computation – just do that for as long as you need, until you get to the last 28 bytes.

Sample:

public byte[] Sha1ExceptEnd(Stream input, int bytesToOmit)
{
    long bytesToRead = input.Length - bytesToOmit;
    byte[] buffer = new byte[16 * 1024]; // Hash up to 16K at a time
    using (SHA1 sha1 = SHA1.Create())
    {
        while (bytesToRead > 0)
        {
            int thisPass = (int) Math.Min(buffer.Length, bytesToRead);
            int bytesReadThisPass = input.Read(buffer, 0, thisPass);
            if (bytesReadThisPass <= 0)
            {
                throw new IOException("Unexpected end of data");
            }  
            sha1.TransformBlock(buffer, 0, bytesReadThisPass, buffer, 0);
            bytesToRead -= bytesReadThisPass;
        }
        // Flush the hash
        sha1.TransformFinalBlock(buffer, 0, 0);
        return sha1.Hash;
    }
}