Question

I have one process that’s gathering data in real time and another process that’s plotting it. The two processes are connected by a pipe, with the data acquisition process feeding the data plotting process.

I value the speed of the data acquisition part more than the reliability of the plotting part. A quick skim shows that the default behavior in a pipe is for the writing and reading ends of a pipe to exhibit blocking behavior, if the partner process is slower. This is bad because the data acquisition process can wait on the plotting process.

Is there a way to have a shell pipe be nonblocking, a la C’s O_NONBLOCK? I don’t care if one data point doesn’t get plotted because it gets overwritten by a newer point…

EDIT: Actually, I think the pipe buffer is large enough to hold data acquisition process’s output without the plotting part immediately needing to process it.

Answer 1

If the data acquisition process needs to run unfettered by the plotting process, you need a different structure than a pipe connecting the two – or you need an extra process in the middle that can discard old data points that have not yet been sent to the plotter.

Conceptually:

+-------------+    +------------+    +-------+
| Acquisition |--->|Holding Tank|--->|Plotter|
+-------------+    +------------+    +-------+

The ‘holding tank’ can be using select() or a similar call on its input and output descriptors, and can hold a finite quantity of data points. When a new point arrives from the acquisition process, if its holding tank is full, it can discard the oldest point and add the new one in its place. If there is space in the plotters input, it can write a new data point to the plotter. Meanwhile, the acquisition process can run flat out (as long as the holding tank never blocks it), and the plotter can run flat out too. These three processes are all connected by pipes. None of the pipes is run as a non-blocking operation.