Question

I have a simple python script that just takes in a filename, and spits out a modified version of that file. I would like to redirect stdout (using ‘>’ from the command line) so that I can use my script to overwrite a file with my modifications, e.g. python myScript.py test.txt > test.txt

When I do this, the resulting test.txt does not contain any text from the original test.txt – just the additions made by myScript.py. However, if I don’t redirect stdout, then the modifications come out correctly.

To be more specific, here’s an example:

---

myScript.py:

#!/usr/bin/python
import sys

fileName = sys.argv[1]
sys.stderr.write('opening ' + fileName + '\n')

fileHandle = file(fileName)
currFile = fileHandle.read()
fileHandle.close()

sys.stdout.write('MODIFYING\n\n' + currFile + '\n\nMODIFIED!\n')

---

test.txt

Hello World

---

Result of python myScript.py test.txt > test.txt:

MODIFYING



MODIFIED!

Answer 1

The reason it works that way is that, before Python even starts, Bash interprets the redirection operator and opens an output stream to write stdout to the file. That operation truncates the file to size 0 – in other words, it clears the contents of the file. So by the time your Python script starts, it sees an empty input file.

The simplest solution is to redirect stdout to a different file, and then afterwards, rename it to the original filename.

python myScript.py test.txt > test.out && mv test.out test.txt

Alternatively, you could alter your Python script to write the modified data back to the file itself, so you wouldn’t have to redirect standard output at all.