Question

Let’s say I’m receiving a file over a socket stream, I am receiving 1024 bytes at a time. Each time I write to the harddisk, my antivirus scans the entire file. The bigger the file gets, the longer it takes to write the next 1024 bytes. Not to mention the “file is in use by another process” errors.

My workaround at the moment is to just store the bytes in a byte array in the memory, up to X megabytes (user defined), the byte array is appended to the file on the harddisk every time it fills up.

byte[] filebytearray = new byte[filesize]; //Store entire file in this byte array.

do
{
    serverStream = clientSocket.GetStream();
    bytesRead = serverStream.Read(inStream, 0, buffSize); //How many bytes did we just read from the stream?
    recstrbytes = new byte[bytesRead]; //Final byte array this loop
    Array.Copy(inStream, recstrbytes, bytesRead); //Copy from inStream to the final byte array this loop
    Array.Copy(recstrbytes, 0, filebytearray, received, bytesRead); //Copy the data from the final byte array this loop to filebytearray

    received += recstrbytes.Length; //Increment bytes received

}while (received < filesize);

addToBinary(filebytearray, @"C:\test\test.exe"); //Append filebytearray to binary

(In this simplified example it just stores the entire file in memory before unloading it to hdd)

But I absolutely hate this method because it significantly increases the memory my program uses.

How do other programmers tackle this issue? When I download with firefox, as an example, it just downloads with full speed, my AV doesn’t seem to pick it up until it’s done, and it barely increases the process’ memory usage. What’s the big secret here?

Append to binary function I am using (WIP):

private bool addToBinary(byte[] msg, string filepath)
{
    Console.WriteLine("Appending "+msg.Length+" bytes of data.");

    bool succ = false;

    do
    {
        try
        {
            using (Stream fileStream = new FileStream(filepath, FileMode.Append, FileAccess.Write, FileShare.None))
            {
                fileStream.Write(msg, 0, msg.Length);
                fileStream.Flush();
                fileStream.Close();
            }
            succ = true;
        }
        catch (IOException ex) { /*Console.WriteLine("Write Exception (addToBinary) : " + ex.Message);*/ }
        catch (Exception ex) { Console.WriteLine("Some Exception occured (addToBinary) : " + ex.Message); return false; }
    } while (!succ);
    return true;
}

Answer 1

I see that you reopen the file every time you write data. Why not keep the file stream opened? Every time you close it, the antivirus scans it, because it was modified.

And one suggestion, the WriteLine function works like printf in c++, so… Instead of doing:

Console.WriteLine("Appending "+msg.Length+" bytes of data.");

you could do:

Console.WriteLine("Appending {0} bytes of data.", msg.Length);

This could really save your time sometimes.