I have a simple command-line utility which produces output both on the console and the filesystem. While I know very well how to capture the console output, I am not aware how can I also intercept the file – for which I know the filename in advance.
I would like to keep the execution “in memory” without touching the filesystem as I immediately parse and delete the file created and this creates an unnecessary bottleneck (especially when I need to run the little tool millions of times).
So, to sum up, I am trying to achieve following:
- Run a binary using python’s subprocess
- Capture both the tool’s output AND contents of a file it creates (in current working directory with in-advance known name)
- Ideally, run it all without touching the filesystem.
Since you only need to support Linux, one possibility is to use named pipes. The idea is to pre-create the output file as a named pipe, and have your process read the tool’s output from the pipe.
See, for example, Introduction to Named Pipes.
The Python API is
os.mkfifo().