Question

I’m writing a distributed system in c++ using TCP/IP and sockets.

For each of my messages, I need to receive the first 5 bytes to know the full length of the incoming message.

What’s the best way to do this?

1. recv() only 5 bytes, then recv() again. if I choose this, would it be safe to assume I’ll get 0 or 5 bytes in the recv (aka not write a loop to keep trying)?
2. use MSG_PEEK
3. recv() some larger buffer size, then read the first 5 bytes and allocate the final buffer then.

Answer 1

You don’t need to know anything. TCP is a stream protocol, and at any given moment you can get as little as one byte, or as much as multiple megabytes of data. The correct and only way to use a TCP socket is to read in a loop.

char buf[4096];        // or whatever

std::deque<char> data;

for (int res ; ; )
{
    res = recv(fd, buf, sizeof buf, MSG_DONTWAIT);

    if (res == -1)
    {
        if (errno == EAGAIN || errno == EWOULDBLOCK)
        {
            break;  // done reading
        }
        else
        {
            // error, break, die
        }
    }
    if (res == 0)
    {
        // socket closed, finalise, break
    }
    else
    {
        data.insert(data.end(), buf, buf + res);
    }
}

The only purpose of the loop is to transfer data from the socket buffer to your application. Your application must then decide independently if there’s enough data in the queue to attempt extraction of some sort of higher-level application message.

For example, in your case you would check if the queue’s size is at least 5, then inspect the first five bytes, and then check if the queue holds a complete application message. If no, you abort, and if yes, you extract the entire message and pop if off from the front of the queue.