Question

I am reading a single data item from a UDP port. It’s essential that this read be the lowest latency possible. At present I’m reading via the boost::asio library’s async_receive_from method. Does anyone know the kind of latency I will experience between the packet arriving at the network card, and the callback method being invoked in my user code?

Boost is a very good library, but quite generic, is there a lower latency alternative?

All opinions on writing low-latency UDP network programs are very welcome.

EDIT: Another question, is there a relatively feasible way to estimate the latency that I’m experiencing between NIC and user mode?

Answer 1

Your latency will vary, but it will be far from the best you can get. Here are few things that will stand in your way to the better latency:

Boost.ASIO

1. It constantly allocates/deallocates memory to store “state” in order to invoke a callback function associated with your read operation.
2. It does unnecessary mutex locking/unlocking in order to support a broken mix of async and sync approaches.
3. The worst, it constantly adds and removes event descriptors from the underlying notification mechanism.

All in all, asio is a good library for high-level application developers, but it comes with a big price tag and a lot of CPU cycle eating gremlins. Another alternative is libevent, it is a lot better, but still aims to support many notification mechanisms and be platform-independent. Nothing can beat native mechanisms, i.e. epoll.

Other things

1. UDP stack. It doesn’t do a very good job for latency sensitive applications. One of the most popular solutions is OpenOnload. It by-passes the stack and works directly with your NIC.
2. Scheduler. By default, scheduler is optimized for throughput and not latency. You will have to tweak and tune your OS in order to make it latency oriented. Linux, for example, has a lot of “rt” patches for that purpose.
3. Watch out not to sleep. Once your process is sleeping, you will never get a good wakeup latency compared to constantly burning CPU and waiting for a packet to arrive.
4. Interference with other IRQs, processes etc.

I cannot tell you exact numbers, but assuming that you won’t be getting a lot of traffic, using Boost and a regular Linux kernel, with a regular hardware, your latency will range somewhere between ~50 microseconds to ~100 milliseconds. It will improve a bit as you get more data, and after some point start dropping, and will always be ranging. I’d say that if you are OK with those numbers, don’t bother optimizing.