Question

Is there a big performance difference between:

• Process A writing to a temp file, and process B reading that file
• Process A writing to a pipe, and process B reading from that pipe

I’m curious to know what the answer is for both Windows and *nix.

EDIT: I should have asked: Does the buffer cache eliminate the difference between a temp file and a pipe?

Answer 1

One big difference is that with the pipe, processes A and B can be running concurrently, so that B gets to work on the output from A before A has finished producing it. Further, the size of the pipe is limited, so A won’t be able to produce vastly more data than B has consumed; it will be made to wait for B to catch up.

If the volume of data is big, then writing to the temporary file involves disk activity, even if only for creating and then destroying the file. The data might well stay in the in-memory buffer pools – so no disk I/O there – even for surprisingly large files. Writing to the pipe ‘never’ involves writing to disk.