Question

Consider a cache application with thousands of requests per second where the client asks for an object using an integer index and the server sends the object back to it. (Linux only)

What would be faster for an UDP protocol:

• Send multiple replies in a single packet?
• Send a single reply in one packet?

(applicable for both sides , server or client)

Disadvantages:

• Multiple replies per packet:

  • many memcpy()s will be needed to assemble packet with multiple replies. If a get() request returns to the client 16 byte objects, a single packet of 65535 bytes would have to memcpy() about 4000 objects before issuing sendto() system call. memcpy() is costly
  • An UDP packet of 65000 bytes will generate about 43 udp packet headers
    since the packet will be transmitted in fragmetns, due to 1500 default MTU size. not a big overhead
• Single reply per packet:
  • every get() will result in 28 bytes of protocol overhead. For an object of 16 bytes this will be costly. The throughput of objects per second will be low and network will be saturated with packets. For example, sending 4000 objects will generate 4000 udp packet headers, condierably more than 43 headers with multiple replies.
  • Also, too many sendto() syscalls will increase CPU consumption becase the kernel has to save CPU registers on each syscall.

It appears that both methods have their disadvantages and it is not clear which solution will be better, however, the clock of gigabit ethernet runs at 1GHz while the processor is for example 3 GHz also the memory
bus is much wider than the twisted pair of the ethernet interface. So, multiple replies
per packet would be better choice since the extra memcpy()s will reduce network traffic, correct?

Now, if we use TCP instead of UDP:

• Multiple replies per packet:
  • again, many memcpy()s will be needed to assemble the packet, which shouldnt be larger than 65535 and then it would be fragmented by the kernel to send it in packets of size of the MTU not much difference as with UDP here
• Single reply per packet:
  • Since our 16 byte objects are distributed randomly in server’s memory, we will have to issue a write() syscall per object, the same number of syscalls as with udp single request case.
  • every get() will result in 52 bytes of overhead due to that TCP has larger header +
    we will have to another overhead of ACK packet which is abour 40 bytes. even bigger network traffic

Conclusion:

1. For this particular application (sending simple data chunk to the client from an index given by the client) TCP will not perform any better than UDP. + consider that the implementation of TCP stack is much more complicated than UDP, more instructions are executed there. The whole point for making the protocol to run faster is to assembly UDP packets as large to the MTU as possible.
2. The cost of adding another NIC to the system is lower than adding another CPU. It makes sense to go for lowering the number of memcpy()s but send a bunch of small UDP packets and pay extra for the overhead of each udp header on the network side (+ some little more overhead by doing many sendto() syscalls which i think will be lower than memcpy()’s overhead) because this way with one CPU could send requests over many NICs

Will greatly appreciate your comments and your experiences on the choice of protocol.

Note:
lets set aside the reliability issues and assume our networks are error free, which is
most of the time true if you wire them correctly, and if an error occurs it is certainly not the norm

Answer 1

There are a few statements here which are not accurate and change the picture a bit:

1. Multiple objects do not require multiple write() system calls, you can use writev() which fits this exact need. (good for both UDP and TCP)

2. Sending a 65K byte buffer with UDP will not create multiple headers of UDP, only multiple headers of IP with a single UDP header. Furthermore, with today 1Gbit NIC/s you could use jambo frames and increase the MTU (beyond 1500) if this is for a private network.

3. For TCP, sending an object with a write()/Send() will not generate a packet necessarily, most likely it will wait until you fill up the MSS and then dispatch several objects, remember that it is a stream and it has no packet boundaries as UDP has.

4. You cannot assume reliability, order and load, making UDP harder to manage in my opinion. TCP will handle connections when servers or clients start to be overloaded with traffic. With UDP you will have to detect such cases and react.