Question

I would like to run a process with subprocess.Popen() and communicate with it through the python shell, like the subprocess.Popen usual behavior. Beside that, I would like to discursively log the STDIN and STDOUT to a logfile.

How can I do it?

Answer 1

Assuming discursively means rambling and rambling means all in the same file, then the following snippet is what you requested.

Discursive logging with discrimination of the source and interaction

Override its communicate method like similar question here

import subprocess

def logcommunicate(self, s):
    self.logfilehandle.write("Input "+s)
    std = self.oldcommunicate(s)

    self.logfilehandle.write("Output "+std[0])
    return std

subprocess.Popen.oldcommunicate = subprocess.Popen.communicate
subprocess.Popen.communicate = logcommunicate
logfh = open("/tmp/communicate.log", "a")

proc = subprocess.Popen(['cat'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
proc.logfilehandle = logfh

result = proc.communicate("hello there\n")
print result 

Discursive logging with discrimination of the source

First use StringIO instead of files, then subclass StringIO to override its write method to open that appends timestamp and source. Then write a custom compare function that sorts based on timestamp and source, timestamp first and then source Input and then output

 with open("file.log","wb") as in logfile:
 out = MyOutPutStringIO.StringIO() 
 in = MyInputStringIO.StringIO()
 subprocess.Popen(cmd, shell=True, universal_newlines = True, stdin=in, stdout=out)

 #Then after you are done
 linestotal = []
 for line in in.readlines():
     linestotal.append(line)
 for line in out.readlines():
     linestotal.append(line)

 linestotal.sort(customsortbasedontimestampandinput)

 for line in linestotal.readlines():
    logwrite.write(line)

Discursive logging

 with open("file.log","wb") as in logfile:
 subprocess.Popen(cmd, shell=True, universal_newlines = True, stdin=logfile, stdout=logfile)

The opposite is shown below

Cursive logging

 with open("stdout.txt","wb") as out:
 with open("stderr.txt","wb") as err:
 with open("stdin.txt","wb") as in:
 subprocess.Popen(cmd, shell=True, universal_newlines = True, stdin=in,stdout=out,stderr=err)