Question

How would one call a shell command from Python which contains a pipe and capture the output?

Suppose the command was something like:

cat file.log | tail -1

The Perl equivalent of what I am trying to do would be something like:

my $string = `cat file.log | tail -1`;

Answer 1

Use a subprocess.PIPE, as explained in the subprocess docs section “Replacing shell pipeline”:

import subprocess
p1 = subprocess.Popen(["cat", "file.log"], stdout=subprocess.PIPE)
p2 = subprocess.Popen(["tail", "-1"], stdin=p1.stdout, stdout=subprocess.PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output,err = p2.communicate()

Or, using the sh module, piping becomes composition of functions:

import sh
output = sh.tail(sh.cat('file.log'), '-1')