Question

This question is related to A Django Related Question

I am running a restlite WSGI app with wsgiref.simple_server. I have this set up so that before the serve_forever() method gets calls it initializes some objects. Most relevantly these classes.

import Queue
import threading
import Deployer
import ParallelRunner
import sys
import subprocess

class DeployManager:
def __init__(self):
    self.inputQueue = Queue.Queue() 
    self.workerThread = DeployManagerWorkerThread(self.inputQueue)
    self.workerThread.start()   
def addDeployJob(self, appList):
    self.inputQueue.put(appList)  #make sure this handles the queue being full
def stopWorker(self):
    self.workerThread.running = False
def __del__(self):
    self.stopWorker()

class DeployManagerWorkerThread(threading.Thread):
def __init__(self, Queue):
    super(DeployManagerWorkerThread, self).__init__()
    self.queue = Queue
    self.running = True
    self.deployer = Deployer.Deployer()
    self.runner = ParallelRunner.ParallelRunner()

def run(self):
    while self.running:
        try:
            appList = self.queue.get(timeout = 10) #This blocks until something is in the queue
            sys.stdout.write('Got deployment job\n')
                            command = "ssh " + server.sshUsername + "@" + server.hostname + "" + " -i " + server.sshPrivateKeyPath + r" 'bash -s' < " + pathToScript 
                            self.process = subprocess.Popen(command,shell=True ,stdin=subprocess.PIPE, stdout=subprocess.PIPE)
            output = process.communicate()[0]
            sys.stdout.write(output + '\n')
        except Queue.Empty:
            pass
    sys.stdout.write('DeployManagerWorkerThread exiting\n')

The restlite request is set up like this

@restlite.resource
def deployAll():
def POST(request, entity):
    GlobalDeployManager.addDeployJob(GlobalAppList) #TODO should runners be configurable at start up
    #print output                                               #or on a run by run basis
    return 'Job added'
return locals()

Which would put an entry into the queue which the workerThread would then grab and start processing. However the call to POpen always hangs I stepped into this and it appears it hangs at a call to os.fork() which will “freeze” the server. When the thread gets to the POpen command the main thread is at the line where it is accepting new requests. I believe the server is using epoll for this. If there are multiple jobs in the queue i can press control-C at the console the server is running in and the main thread will exit (Not be at the line which is waiting for requests) and then the thread will be able to run will work as expected and then close. Any ideas?

Answer 1

I eventually figured this out… I needed to create the threads object in the run method. I accidentally forgot that the init method was called from the main thread and then the new thread itself never executes init.