Question

I have two separate subprocess calls in a django app, each in a different view. The first calls the dcraw image conversion tool to take an uploaded raw image and convert it into a tiff. (The .tiff output is usually nine times larger than the input, e.g. 8mb image yields 72mb tiff.)

pdcraw = subprocess.Popen(dcraw_args, stdout=None, stderr=None)
pdcraw.communicate()

My second subprocess calls a matlab script which processes the tiff and outputs relevant data files and images. This call usually takes a good 15-20 seconds when I run the script myself.

result = subprocess.Popen(matlab_args, stdout=None, stderr=None)
result.communicate()

My problem: with both of these subprocess calls, I have to reload the view manually for them to actually run. The rest of the code “around” them runs fine. Any thoughts on how I can avoid this? I’m wondering if it’s some sort of cache issue, as I have been able to get the first subprocess to run a couple times before it starts requiring a reload. Your insight is greatly appreciated! Thank you in advance.

Things I’ve tried:

• Inserting time.sleep around the subprocesses, thinking maybe it’s cut short. Doesn’t work.
• Using pipes instead of None for stdout. I figure I should use None though, since I don’t need to read the results from the command (they just output images to a directory specified in the args).
• Cursing at it. Still in progress.

Answer 1

because the whole idea of a subprocess is being asynchronous, it won’t be ready when your view is parsed to the browser.

Perhaps you make an Ajax request to check the status of the progress, and as soon as files are available, they can then be send by the server.

Or you work with WebSockets, which can send any data as soon as it is available, omitting the need for extra requests while guaranteeing a superquick response.