Question

I have a script that produces a lot of output. The script pauses for a few seconds at point T.

Now I am using the less command to analyze the output of the script.
So I execute ./script | less. I leave it running for sufficient time so that the script would have finished executing.

Now I go through the output of the less command by pressing Pg Down key. Surprisingly while scrolling at the point T of the output I notice the pause of few seconds again.

The script does not expect any input and would have definitely completed by the time I start analyzing the output of less.

Can someone explain how the pause of few seconds is noticable in the output of less when the script would have finished executing?

Answer 1

Your script is communicating with less via a pipe. Pipe is an in-memory stream of bytes that connects two endpoints: your script and the less program, the former writing output to it, the latter reading from it.

As pipes are in-memory, it would be not pleasant if they grew arbitrarily large. So, by default, there’s a limit of data that can be inside the pipe (written, but not yet read) at any given moment. By default it’s 64k on Linux. If the pipe is full, and your script tries to write to it, the write blocks. So your script isn’t actually working, it stopped at some point when doing a write() call.

How to overcome this? Adjusting defaults is a bad option; what is used instead is allocating a buffer in the reader, so that it reads into the buffer, freeing the pipe and thus letting the writing program work, but shows to you (or handles) only a part of the output. less has such a buffer, and, by default, expands it automatically, However, it doesn’t fill it in the background, it only fills it as you read the input.

So what would solve your problem is reading the file until the end (like you would normally press G), and then going back to the beginning (like you would normally press g). The thing is that you may specify these commands via command line like this:

./script | less +Gg

You should note, however, that you will have to wait until the whole script’s output loads into memory, so you won’t be able to view it at once. less is insufficiently sophisticated for that. But if that’s what you really need (browsing the beginning of the output while the ./script is still computing its end), you might want to use a temporary file:

 ./script >x & less x ; rm x