Question

I have the following code in a loop:

while true:
    # Define shell_command
    p1 = Popen(shell_command, shell=shell_type, stdout=PIPE, stderr=PIPE, preexec_fn=os.setsid)
    result = p1.stdout.read(); 
    # Define condition
    if condition:
       break;

where shell_command is something like ls (it just prints stuff).

I have read in different places that I can close/terminate/exit a Popen object in a variety of ways, e.g. :

p1.stdout.close()
p1.stdin.close()
p1.terminate
p1.kill

My question is:

1. What is the proper way of closing a subprocess object once we are done using it?
2. Considering the nature of my script, is there a way to open a subprocess object only once and reuse it with different shell commands? Would that be more efficient in any way than opening new subprocess objects each time?

Update

I am still a bit confused about the sequence of steps to follow depending on whether I use p1.communicate() or p1.stdout.read() to interact with my process.

From what I understood in the answers and the comments:

If I use p1.communicate() I don’t have to worry about releasing resources, since communicate() would wait until the process is finished, grab the output and properly close the subprocess object

If I follow the p1.stdout.read() route (which I think fits my situation, since the shell command is just supposed to print stuff) I should call things in this order:

1. p1.wait()
2. p1.stdout.read()
3. p1.terminate()

Is that right?

Answer 1

What is the proper way of closing a subprocess object once we are done using it?

stdout.close() and stdin.close() will not terminate a process unless it exits itself on end of input or on write errors.

.terminate() and .kill() both do the job, with kill being a bit more “drastic” on POSIX systems, as SIGKILL is sent, which cannot be ignored by the application. Specific differences are explained in this blog post, for example. On Windows, there’s no difference.

Also, remember to .wait() and to close the pipes after killing a process to avoid zombies and force the freeing of resources.

A special case that is often encountered are processes which read from STDIN and write their result to STDOUT, closing themselves when EOF is encountered. With these kinds of programs, it’s often sensible to use subprocess.communicate:

>>> p = Popen(["sort"], stdin=PIPE, stdout=PIPE)
>>> p.communicate("4\n3\n1")
('1\n3\n4\n', None)
>>> p.returncode
0

This can also be used for programs which print something and exit right after:

>>> p = Popen(["ls", "/home/niklas/test"], stdin=PIPE, stdout=PIPE)
>>> p.communicate()
('file1\nfile2\n', None)
>>> p.returncode
0

Considering the nature of my script, is there a way to open a subprocess object only once and reuse it with different shell commands? Would that be more efficient in any way than opening new subprocess objects each time?

I don’t think the subprocess module supports this and I don’t see what resources could be shared here, so I don’t think it would give you a significant advantage.