Question

After enough playing with asynchronous socket programming I noticed that the server was receiving chunked payloads (ie: more than one complete payload sitting in the same buffer). So I came up with the following:

if (bytes_to_read > 0)
{
    while (bytes_to_read > 0)

        // Get payload size as int.

        // Get payload in byte format.

        // Do something with payload.

        // Decrease the amount of bytes to read.
    }

    // Wait for more data.
}

And then I noticed packet fragmentation (ie: what I thought were complete payloads chunked together wasn’t always so) which changed the previous code to something like:

if (bytes_to_read > 0)
{
    while (bytes_to_read > 0)
    {
        // Get payload size as int.

        // Check if the payload size is less than or equal to the amount of bytes left to read.
        if (payload_size <= bytes_to_read)
        {
            // Get payload in byte format.

            // Do something with payload.

            // Decrease the amount of bytes to read.
        }
        else
        {
            // We received a fragmented payload.
            break;
        }
    }

    if (bytes_to_read == 0)
    {
        // Wait for more data.
    }
    else if (bytes_to_read > 0)
    {
        // Wait for more data where we left off. ***
    }
    else
    {
        // Something awful happened.
    }
}

*** I don’t even know how to go about this and would like to see code for it. I had an idea that it involved copying the in-completed payload to the beginning of the buffer and then picking up from there.

The pseudo code I included is based on the Begin* End* method I am using (I’m aware that I should be using the *Async set of methods found here -> http://msdn.microsoft.com/en-us/library/system.net.sockets.socketasynceventargs.aspx <- but I think my overall question still applies).

I am seeking the answers to 2 questions–namely:

1. Is this approach correct or am I
   missing something?
2. What does a working example of
   dealing with packet fragmentation in
   C# look like?

EDIT: I’m using raw sockets.

Thanks in advance for all your help.

EDIT: John Saunders and Greg Hewgill have brought up the point of treating data as a stream but that does not provide me with a concrete example of how to deal with the last chunked payload sometimes being fragmented.

EDIT: I have read Jon Skeet’s answer here which is basically along the same lines as the other answers I have seen but it doesn’t help me much as I already get what I have to do but not how to do it.

EDIT: To elaborate on what I mean by fragmentation, consider the following the receive buffers:

• 224TEST3foo3bar
• 224TEST3foo3bar224TEST3foo3bar
• 224TEST3foo3bar224TEST3foo
• 3bar224TEST3foo3bar

EDIT: I found this and this which lead me here. Vadym Stetsiak has cleared nearly everything up (his was one of the answers I was looking for).

Answer 1

When you have to do it yourself, it can be done like so (reference here):

/// 
/// Server state holds current state of the client socket
///
class AsyncServerState
{
   public byte[] Buffer = new byte[512]; //buffer for network i/o
   public int DataSize = 0; //data size to be received by the server

   //flag that indicates whether prefix was received
   public bool DataSizeReceived = false;

   public MemoryStream Data = new MemoryStream(); //place where data is stored
   public SocketAsyncEventArgs ReadEventArgs = new SocketAsyncEventArgs();
   public Socket Client;
}

/// 
/// Implements server receive logic
/// 
private void ProcessReceive(SocketAsyncEventArgs e)
{
    //single message can be received using several receive operation
    AsyncServerState state = e.UserToken as AsyncServerState;

    if (e.BytesTransferred <= 0 || e.SocketError != SocketError.Success)
    {
        CloseConnection(e);
    }

    int dataRead = e.BytesTransferred;
    int dataOffset = 0;
    int restOfData = 0;

    while (dataRead > 0)
    {
        if (!state.DataSizeReceived)
        {
            //there is already some data in the buffer
            if (state.Data.Length > 0)
            {
                restOfData = PrefixSize - (int)state.Data.Length;
                state.Data.Write(state.Buffer, dataOffset, restOfData);
                dataRead -= restOfData;
                dataOffset += restOfData;
            }
            else if (dataRead >= PrefixSize)
            {   //store whole data size prefix
                state.Data.Write(state.Buffer, dataOffset, PrefixSize);
                dataRead -= PrefixSize;
                dataOffset += PrefixSize;
            }
            else
            {   // store only part of the size prefix
                state.Data.Write(state.Buffer, dataOffset, dataRead);
                dataOffset += dataRead;
                dataRead = 0;
            }

            if (state.Data.Length == PrefixSize)
            {   //we received data size prefix
                state.DataSize = BitConverter.ToInt32(state.Data.GetBuffer(), 0);
                state.DataSizeReceived = true;

                state.Data.Position = 0;
                state.Data.SetLength(0);
            }
            else
            {   //we received just part of the headers information
                //issue another read
                if (!state.Client.ReceiveAsync(state.ReadEventArgs))
                    ProcessReceive(state.ReadEventArgs);
                    return;
            }
        }

        //at this point we know the size of the pending data
        if ((state.Data.Length + dataRead) >= state.DataSize)
        {   //we have all the data for this message

            restOfData = state.DataSize - (int)state.Data.Length;

            state.Data.Write(state.Buffer, dataOffset, restOfData);
            Console.WriteLine("Data message received. Size: {0}",
                                  state.DataSize);

            dataOffset += restOfData;
            dataRead -= restOfData;

            state.Data.SetLength(0);
            state.Data.Position = 0;
            state.DataSizeReceived = false;
            state.DataSize = 0;

            if (dataRead == 0)
            {
                if (!state.Client.ReceiveAsync(state.ReadEventArgs))
                    ProcessReceive(state.ReadEventArgs);
                    return;
            }
            else
                continue;
        }
        else
        {   //there is still data pending, store what we've
            //received and issue another BeginReceive
            state.Data.Write(state.Buffer, dataOffset, dataRead);

            if (!state.Client.ReceiveAsync(state.ReadEventArgs))
                ProcessReceive(state.ReadEventArgs);

            dataRead = 0;
        }
    }
}

I did not do it exactly this way myself but it helped.