Question

I have created a script to run a test script on a batch of files,have been testing it overnight for two nights, however it just hangs at a certain point.

I was wondering if the the commands.getstatusoutput() is the issue here since the test script is has a heavy logging mechanism.

Update:
How is using subprocess module functions different from using os.system() which way of doing things is better?

Answer 1

The method getstatusoutput() returns a string, which can be very long and therefore take up a lot of space and cause paging to disk and other nasty things.

Since the commands module is deprecated anyway, better use the subprocess module which provides a file-like access to the process output. If you need the output for later, just write it into a file on the hard disk

cmd = subprocess.Popen(['ls'], stdout=file('output', 'w'))

If you want process to process the output, read stdout like this

cmd = subprocess.Popen(['ls'], stdout=subprocess.PIPE)
for line in cmd.stdout:
    do_stuff(line)

Which will be easier on your memory.