Question

I’m looking for the lowest latency IPC that allow to put one process to sleep and allow other process to wake it.

I’m looking for the lowest latency method. Some possible methods so far:

1. Writing a byte to a pipe and reading it from it.
2. Writing a byte to a socket and reading it from it.
3. Sending a signal (kill) and waiting for it (sigwait)
4. Using sem_post/sem_wait

Any other better ideas?

Any solution that is Linux specific is fine as well.

Answer 1

Generally… There is almost no difference between the OS methods.

Setup:

1. Two processes with affinity to two different CPUs.
2. One process sleeps (nanosleep) for N microseconds measures current time
   and then notifies other process.
3. Other process wakes measures current time and compares it to the client’s time.
4. Average, standard deviation, median and percentile 95 is calculated over 1K samples after warm-up on 100 notifications.
5. OS: Linux 2.6.35 x86_64
6. CPU: Intel i5 M460

Results:

Semaphore (sem_wait/sem_post – kernel – futex):

sleep us     mean             median      %95
       1     4.98 ±18.7         3.78     5.04
      10     4.14 ±14.8         3.54     4.00
     100    20.60 ±29.4        22.96    26.96
    1000    49.42 ±37.6        30.62    78.75
   10000    63.20 ±22.0        68.38    84.38

Signal (kill/sigwait)

sleep us     mean             median      %95
       1     4.69 ±3.8          4.21     5.39
      10     5.91 ±14.8         4.19     7.45
     100    23.90 ±17.7        23.41    35.90
    1000    47.38 ±28.0        35.27    81.16
   10000    60.80 ±19.9        68.50    82.36

Pipe (pipe + write/read)

sleep us     mean             median      %95
       1     3.75 ±5.9          3.46     4.45
      10     4.42 ±3.5          3.84     5.18
     100    23.32 ±25.6        24.17    38.05
    1000    51.17 ±35.3        46.34    74.75
   10000    64.69 ±31.0        67.95    86.80

Socket (socketpair +write/read)

sleep us     mean             median      %95
       1     6.07 ±3.2          5.55     6.78
      10     7.00 ±7.1          5.51     8.50
     100    27.57 ±14.1        28.39    50.86
    1000    56.75 ±25.7        50.82    88.74
   10000    73.89 ±16.8        77.54    88.46

As a reference busy waiting:

sleep us     mean             median      %95
       1     0.17 ±0.5          0.13     0.23
      10     0.15 ±0.3          0.13     0.19
     100     0.17 ±0.3          0.16     0.21
    1000     0.22 ±0.1          0.18     0.35
   10000     0.38 ±0.3          0.30     0.78