Question

I am going through a big list of files and running a command on all of them. I want to log the output and if the command takes more than 5 minutes for a file, I would like to stop the command and go to the next file.

I have two problems:

1. I want to record the file name to the output file and also record the output messages. I am using Popen to log the messages and using communicate so it gets logged, but all the filenames I am writing with write() don’t get written till the whole task is done.

2. I am not sure how I can poll the process and quit it after 5 minutes and go to the next file.

Below is simplified code:

import os, fnmatch
import subprocess
import sys
f=open('filenames','w')

'Locate all files matching supplied filename pattern in and below supplied root directory.'''

def locate(pattern, root=os.curdir):        
    for path, dirs, files in os.walk(os.path.abspath(root)):
        for filename in fnmatch.filter(files, pattern):
            yield os.path.join(path, filename)


for filename in locate("*.dll"):
    f.write(filename)   
    #cmd defintion is changed for simplicity 
    cmd='cat %s' %filename
    p=subprocess.Popen(cmd,stdout=f)   
    p.communicate()[0]

Answer 1

1. Use f.flush() after f.write(filename).
2. See Using module 'subprocess' with timeout