Question

Apologies if this kind of thing has been answered elsewhere. I am using Python to run a Windows executable file using subprocess.Popen(). The executable file produces a .txt file and some other output files as part of its operation. I then need to run another executable file using subprocess.Popen() that uses the output from the original .exe file.

The problem is, it is the .exe file and not Python that is controlling the creation of the output files, and so I have no control over knowing how long it takes the first text file to write to disk before I can use it as an input to the second .exe file.

Obviously I cannot run the second executable file before the first text file finishes writing to disk.

subprocess.wait() does not appear to be helpful because the first executable terminates before the text file has finished writing to disk. I also don’t want to use some kind of function that waits an arbitrary period of time (say a few seconds) then proceeds with the execution of the second .exe file. This would be inefficient in that it may wait longer than necessary, and thus waste time. On the other hand it may not wait long enough if the output text file is very large.

So I guess I need some kind of listener that waits for the text file to finish being written before it moves on to execute the second subprocess.Popen() call. Is this possible?

Any help would be appreciated.

---

UPDATE (see Neil’s suggestions, below)

The problem with os.path.getmtime() is that the modification time is updated more than once during the write, so very large text files (say ~500 Mb) require a relatively large wait time in between os.path.getmtime() calls. I use time.sleep() to do this. I guess this solution is workable but is not the most efficient use of time.

On the other hand, I am having bigger problems with trying to open the file for write access. I use the following loop:

while True:
    try:
        f = open(file, 'w')
    except:
        # For lack of something else to put in here
        # (I don't want to print anything)
        os.path.getmtime(file) 
    else:
        break

This approach seems to work in that Python essentially pauses while the Windows executable is writing the file, but afterwards I go to use the text file in the next part of the code and find that the contents that were just written have been wiped.

I know they were written because I can see the file size increasing in Windows Explorer while the executable is doing its stuff, so I can only assume that the final call to open(file, ‘w’) (once the executable has done its job) causes the file to be wiped, somehow.

Obviously I am doing something wrong. Any ideas?

Answer 1

There’s probably many ways to do what you want. One that springs to mind is that you could poll the modification time with os.path.getmtime(), and see when it changes. If the modification date is after you called the executable, but still a couple seconds ago, you could assume it’s done.

Alternatively, you could try opening the file for write access (just without actually writing anything). If that fails, it means someone else is writing it.