Question

I am trying to parse a growing csv file with the following script parse.py:

import csv
import sys

reader = csv.reader(sys.stdin)
for row in reader:
    print row

The corresponding command line is tail -F log | python parse.py.

Then I try to append some rows to log with echo "something" >> log.

But parse.py print nothing.

What’s wrong with my script?

Answer 1

The problem you’re having is that what you’re adding to the log file is being buffered and not passed to the other commands in the pipeline.

For more information, please have a look at how big is the pipe buffer.