Question

I need to encrypt and store uploaded files.

These files would then be decrypted and downloaded via a token/ticken mechanism.

It is important that the files do not reside unencrypted, system admins shouldn’t be able to casually access them.

My problem is the files can be quite large, 10 gigs is about the max file size expected.

The encryption process can take as long as needs be.

However I would like the encryption process to work on the fly – so there is no 10 gig decrypted file, only a chunk of it in memory (I don’t have 10 gigs of ram for this).

Any suggestions how to achieve this?

Answer 1

Most encryption is stream-based, so that shouldn’t be a problem at all – just run the FileStream as an input to the CryptoStream, and use that as the source for however you are consuming the data; which could be in-process processing, or could be a destination FileStream (in which case, just cryptoStream.CopyTo(outputFileStream) should suffice).

If I borrow the example from MSDN and edit it to show writing binary:

using(FileStream fStream = File.Open(FileName, FileMode.OpenOrCreate))
{
    // Create a new Rijndael object.
    Rijndael RijndaelAlg = Rijndael.Create();

    // Create a CryptoStream using the FileStream 
    // and the passed key and initialization vector (IV).
    using(CryptoStream cStream = new CryptoStream(fStream,
        RijndaelAlg.CreateDecryptor(Key, IV),
        CryptoStreamMode.Read))
    using(FileStream destination = File.Create(destinationPath))
    {
        cStream.CopyTo(destination);
    }
}

However, cStream could be used for any Stream-based reading.