Question

Possible Duplicate:
Is the received stream from a socket limited to a single send command?

Note: I see this question very complicated (hopefully not for you guys, that’s why Im asking here lol) and I tried my best to explain it as simple and clear as possible.

In my application, I’m continually receiving byte arrays in a fix sized buffer.

These series of byte arrays that I’m receiving has been serialized ‘binarily’.

However, sometimes the byte array received will be bigger than the fix sized buffer so I would need to store the current received byte array into a container and loop again to receive the remaining byte arrays coming in.

My question now is how to “concatenate” or “combine” or “join” all the “batches” of byte arrays I received ( and is stored in a container, possibly a queue of byte arrays) to form a single byte array and then de-serialize them?

int bytesRead = client.EndReceive(ar);
if (bytesRead > 0)
    {
        // There might be more data, so store the data received so far.
        // If the buffer was not filled, I have to get the number of bytes received as Thorsten Dittmar was saying, before queuing it
        dataReceivedQueue.Enqueue(state.buffer);

        // Get the rest of the data.
        client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
        new AsyncCallback(ReceiveCallback_onQuery), state);
    }
else
{
    // All the data has arrived; put it in response.
    response_onQueryHistory = ByteArrayToObject(functionThatCombinesBytes(dataReceivedQueue));

    // Signal that all bytes have been received.
    receiveDoneQuery.Set();
}

state.buffer is buffer where data are received. buffer is a byte array of size 4096. state is of type StateObject.

ByteArrayToObject(byte []) takes care of deserializing the data received and converting it back to its object form

functionThatCombinesBytes(Queue) this function will receive a Queue of bytes and will “combine” all the bytes into one byte array

Answer 1

Just because you are calling BeginReceive with a buffer of a particular size, doesn’t mean that it will necessarily entirely fill the buffer, so it’s very likely that some of your queued buffers will actually only be partially filled with received data, and the remainder being zeros, this will almost certainly corrupt your combined stream if you simply concatenate them together since you’re not also storing the number of bytes actually read into the buffer. You also appear to be reusing the same buffer each time, so you’ll just be overwriting already-read data with new data.

I would therefore suggest replacing your dataReceivedQueue with a MemoryStream, and using something like:

if (bytesRead > 0)
    {
        // There might be more data, so store the data received so far.
        memoryStream.Write(state.buffer, 0, bytesRead);

        // Get the rest of the data.
        client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
        new AsyncCallback(ReceiveCallback_onQuery), state);
    }
else
{
    // All the data has arrived; put it in response.
    response_onQueryHistory = ByteArrayToObject(memoryStream.ToArray());

    // Signal that all bytes have been received.
    receiveDoneQuery.Set();
}