Question

awk newbie here..
I’m trying this:

top -b -p 30259 | awk 'BEGIN { OFS = ","; print "Timestamp,CPU,Memory"} /tomcat/ { print strftime("%H:%M:%S"), $9, $10 }' > asdf.log

But ‘asdf.log’ always stays empty. I tried redirecting to file from within the script:

top -b -p 30259 | awk 'BEGIN { OFS = ","; print "Timestamp,CPU,Memory" > "asdf.log"} /tomcat/ { print strftime("%H:%M:%S"), $9, $10 > "asdf.log" }'

but it still doesn’t work… And printing to stdout is working fine.

What am I doing wrong?

Answer 1

How long are you waiting for data to arrive?

When programs use the standard C library to write to files (fopen(3) and family of functions), the C library will add some buffering to the streams to improve performance. If every read or write of one character actually started a system call, the performance of common programs would be pretty horrible. So the C standard input routines will submit blocks of IO when the buffers are “full enough”.

The tricky parts comes that the C standard IO routines will change their definition of “full enough” based on the specific file descriptors — if the file descriptor is to a file, then the IO is block buffered, and you must wait for a certain amount of data to arrive. (Check stat asdf.log‘s IO Block entry to see the most likely size.) However, if the file descriptor is for stdout and it refers to a terminal device, the output is line buffered — output will be sent to the terminal as soon as a newline character is printed.

Perhaps the setvbuf(3) manpage can explain better than I can:

   The three types of buffering available are unbuffered, block
   buffered, and line buffered.  When an output stream is
   unbuffered, information appears on the destination file or
   terminal as soon as written; when it is block buffered many
   characters are saved up and written as a block; when it is
   line buffered characters are saved up until a newline is
   output or input is read from any stream attached to a
   terminal device (typically stdin).  The function fflush(3)
   may be used to force the block out early.  (See fclose(3).)
   Normally all files are block buffered.  When the first I/O
   operation occurs on a file, malloc(3) is called, and a buffer
   is obtained.  If a stream refers to a terminal (as stdout
   normally does) it is line buffered.  The standard error
   stream stderr is always unbuffered by default.

Amend your script to look like this:

top -b | awk 'BEGIN { OFS = ","; print "Timestamp,CPU,Memory"}
/bash/ { print strftime("%H:%M:%S"), $9, $10; fflush() }' > /tmp/foo

The fflush() forces output immediately. Not great for performance, but better for viewing the state of your system in real time.