Question

Basically the back story is that i’ve built a selecting of python scripts for a client to process importing and exporting batch jobs between their operations database and their ecomms site database. this works fine. these scripts write to stdout to update the user on the status of the batch script.

I’m trying to produce a framework for these scripts to be run via a django view and to post the stdout to the webpage to show the user the progress of these batch processes.

the plan was to
– call the batch script as a subprocess and then save stdout and stderr to a file.
– return a redirect to a display page that will reload every 2 seconds and display line by line the contents of the file that stdout is being written to.

however the problem is, that the stdout/stderr file is not being actually written to until the entire batch script has finished running or errors out.

i’ve tried a number of things, but none seem to work.

heres the current view code.

def long_running(app, filename):
    """where app is ['command', 'arg1', 'arg2'] and filename is the file used for output"""
    # where to write the result (something like /tmp/some-unique-id)
    fullname = temppath+filename
    f = file(fullname, "a+")
    # launch the script which outputs something slowly
    subprocess.Popen(app, stdout=f, stderr=f)# .communicate()
    # once the script is done, close the output
    f.close()

def attributeexport(request):
    filename = "%d_attribute" %(int(time.time())) #set the filename to be the current time stamp plus an identifier
    app = ['python','/home/windsor/django/applications/attribute_exports.py']
    #break thread for processing.
    threading.Thread(target=long_running, args=(app,filename)).start()
    return HttpResponseRedirect('/scripts/dynamic/'+filename+'/')
    pass

def dynamic(request, viewfile):
    fileobj = open(temppath+viewfile, 'r')
    results = []
    for line in fileobj:
        results.append(line)
        if '~END' in line:
            #if the process has completed
            return render_to_response('scripts/static.html', {'displaylist':results, 'filename':viewfile})
    return render_to_response('scripts/dynamic.html', {'displaylist':results, 'filename':viewfile})
pass

Answer 1

It helps if you use the following:

['python','-u','path/to/python/script.py']