Question

I have an “appliance” (for lack of better description) running linux.

Currently I ssh into the box to launch jobs. This isn’t friendly enough for my users, so I’m putting together a simple web UI to launch the script. A job runs for anywhere from 10 seconds to several hours. The web UI needs to reflect the status of the job.

I’ve solved similar problems in the past by running a daemon on the server that watches a spool directory (or db table) for new job requests, spawns a process, monitors the process, and provides info for the web UI in a db table or status file. The web UI then drops job requests into a spool dir (db) and occasionally check the status file (db). This might be overkill for this task.

For the current task, I am considering spawning the job from the cgi and occasionally checking a status file that the job writes as it progresses or exits.

My question: is there a better (simpler/faster-to-write/more robust) way to do this? Are there existing patterns or tools that I should know about?

(Python solutions are ideal.)

Thanks.

Answer 1

I do this in a number of projects. A web-app (mostly Python/CGI) that spawns a separate python script (using subprocess) which instantly daemonizes itself to do the work. The web-app then continues to issue AJAX requests to check on the daemon process progress (I use simple txt files for communication, database would probably be better). One nice touch is to have the daemon email the end user once it finishes (with a link to retrieve results). This way the user can close their web browser on those jobs that take hours.