Question

I’m looking for texts or advice on implementing stop and restart in file stream transfer.

The goal is for the application to use a single read source, and output to multiple write sources, and be able to restart from a recorded position if a transfer fails.

The application is being written in C# .NET.

Psuedo code:

while (reader.Read())
{
    foreach(writer in writers)
    {
        writer.WriteToStream();
    }
}

I need to be able to implement stop or pause. Which could work like so. To stop, continue is marked false:

while (reader.Read() && Continue)
{
    foreach(writer in writers)
    {
        writer.WriteToStream();
    }
}

Clearly at this stage I need to record the number of bytes read, and the number of bytes written to each write source.

My questions are:

If I were to only record the read bytes, and use this for restarts, one or more writers could have written while others have not. Simply restarting using a measure of read progress might corrupt the written data. So I need to use a ‘written bytes per writer’ record as my new start position. How can I be sure that the bytes were written (I may not have the ability to read the file from the write source to read the file length)?

Can anyone adviser or point me in the right direction of a text on this kind of issue?

Answer 1

Use a thread synchronization event.

(pseudocode):

ManualResetEvent _canReadEvent = new ManualResetEvent(true);

public void WriterThreadFunc()
{
    while (_canReadEvent.Wait() && reader.Read())
    {
        foreach(writer in writers)
        {
            writer.WriteToStream();
        }
    }
}

public void Pause()
{
    _canReadEvent.Reset();
}

public void Continue()
{
    _canReadEvent.Set();
}

The good thing is that the writer thread won’t consume any CPU when it’s paused and it will continue directly it’s signaled (as opposed to using a flag and Thread.Sleep())

The other note is that any check should be the first argument in the while since reader.Read() will read from the stream otherwise (but the content will be ignored since the flag will prevent the while block from being executed).