Question

In Windows, that is.

I think the answer to this question is that I need to create a Windows service. This seems ludicrously heavyweight for what I am trying to do.

I’m just trying to slap together a little prototype here for my manager, I’m not going to be responsible for productizing it… in fact, it may never even BE productized; it might just be something that a few researchers play around with.

I have a CGI script that receives a file for upload, stores it to a temporary location, then launches a background process to do some serious number-crunching on the file. Then some Javascript stuff sits around calling other CGI scripts to check on the status and update the page as needed.

All of this works, except the damn web server won’t close the connection as long as the subrocess is running. I’ve done some searching, and it appears the answer on Unix is to make it a daemon, but I’m stuck on Windows right now and I guess the answer there is to make it a Windows service?!? This seems incredibly heavyweight to just, you know, launch a damn process and then close the server connection.

That’s really the only way?

Edit: Okay, found a nifty little hack over here (the choice (3) that the guy gives):
How to completely background a process in Perl CGI under IIS

I was able to modify this to make it even simpler, and although this is a klugey solution, it is perfect for the quick-and-dirty little prototype I am trying to make.

So I initially had my main script doing this:

subprocess.Popen("python.exe","myscript.py","arg1","arg2")

Which doesn’t work, as I’ve described. Instead, I now have my main script emit this little bit of Javascript which runs after the document is fully loaded:

$("#somecrap").load("launchBackgroundProcess.py", {arg1:"foo",arg2:"bar"});

And then launchBackgroundProcess.py does the subprocess.Popen.

This solution would never scale, since it still leaves the browser connection open during the entire time the background task is running. But since this little thinger I am whipping up might someday have two simultaneous users at most (even then I doubt it) resources are not a concern. This allows the user to see the main page and get the Javascript updates even though there is still an http connection hanging open for no good reason.

Thanks for the answers! If I’m ever asked to productize this, I’ll take at the resources Profane recommends.

Answer 1

If you haven’t much experience with windows programming and don’t wish to peruse the MSDN docs– I don’t blame you– you may want to try to pick up a copy of Mark Hammond’s cannonical guide to all things python and windows. It somehow never goes out-of-date on many of these sorts of recurring questions. Instead of launching the process with the every-platform solution, you’d probably be better off using the win32process module. Chapter 17 of the Hammond book covers this extensively, but you could probably get all you need by downloading the pywin ide (I think it comes bundled in the windows extensions which you can download from pypi), and looking through the help docs it has on python’s windows’ api. Here’s an example of using the api, from a project I was working on recently. It may in fact do some of what you want with a little adaptation. You’d probably want to focus on CreationFlags. In particular, win32process.DETACHED_PROCESS is “often used to execute console programs in the background.” Many other flags are available and conveniently wrapped however.

    if subprocess.mswindows:
        su=subprocess.STARTUPINFO()
        su.dwFlags |= subprocess._subprocess.STARTF_USESHOWWINDOW
    process = subprocess.Popen(['program', 'flag', 'flag2'], bufsize=-1,
              stdout=subprocess.PIPE, startupinfo=su)