Question

The subprocess module has the convenience function call, which is implemented like this in both 2.6 and 3.1:

def call(*popenargs, **kwargs):
    return Popen(*popenargs, **kwargs).wait()

The documentation for this function carries a red warning, reading:

Warning: Like Popen.wait(), this will deadlock when using stdout=PIPE and/or stderr=PIPE and the child process generates enough output to a pipe such that it blocks waiting for the OS pipe buffer to accept more data.

The Popen.wait() documentation says to use Popen.communicate() instead in such circumstances. Well, then why isn’t call just implemented like below instead, so the stupid warning can be removed, and silly limitations like this removed from the standard library?

def call(*args, **kwargs):
    input = kwargs.pop("input", None)
    p = Popen(*args, **kwargs)
    p.communicate(input)
    return p.returncode

I’m sure there’s a reason. What am I missing?

Answer 1

I spent some time looking through PEP-324, which introduced the subprocess module, trying to figure out the design decisions involved, but I think the answer is actually very simple:

There’s no reason to pass stdout=PIPE or stderr=PIPE to subprocess.call, so the fact that it can deadlock is irrelevant.

The only reason to pass stdout=PIPE or stderr=PIPE to subprocess.Popen is so that you can use the Popen instance’s stdout and stderr attributes as file objects. Since subprocess.call never lets you see the Popen instance, the PIPE options become irrelevant.

There is potential overhead to Popen.communicate (creating additional threads to avoid deadlock by monitoring the pipes), and there’s no benefit in this case, so there’s no reason to use it.

Edit: If you want to discard your output, I guess it’s better to do so explicitly:

# option 1
with open(os.devnull, 'w') as dev_null:
    subprocess.call(['command'], stdout=dev_null, stderr=dev_null)

# option 2
subprocess.call(['command >& /dev/null'], shell=True)

instead of instructing subprocess to capture all of the output to PIPE files that you never intend to use.