Question

i have tests that i ran which can take up to 15m at a time. during these 15m, a log file is periodically written to. however, most of the content is useless.

in response to this i have a python script that parses out the useless text and displays the relevant data.

what i’m trying to achieve is similar to what tail -f log_file, constantly updating the terminal with the newest additions to a file. i was thinking that if a python script ran as a process, it could parse the log file whenever the tests write to it, then the python script can go to sleep until interrupted again once the log file is written to.

any ideas how one can achieve this?

i already have a script that does the parsing, i just don’t know how to make it do it continually and efficiently.

Answer 1

You could just have the script filter standard input, and pipe tail -f through it. When you’re waiting on stdin, your script will sleep, so it’s plenty efficient.

Eg.

python long_running_script.py && tail -f log_file | python filter_logs.py

Your script can be something like

while true:
    line = sys.stdin.readline()
    if filter_line(line): print line