Question

UPDATED POST
So, I found a solution to what I was doing it is to use Is there any way to kill a Thread in Python?

I would like to get some validation on design. We have a fairly large system, and one components of the system is the communication component. It is in charge of two things, one sending messages, and two queuing any messages that get received (to file,db, queue, etc.) I am receiving the message via a provided library that basically can just wait forever for a message to arrive.

Questions:

1 ) Is it best to have One main thread, and two sub threads?

2) Is it better to allow the Receive thread to just block forever until a message arrives? Or should I allow for it to timeout(which is an exception), and just continue to loop?

Let me know whatever more information you may need.

Right now I basically have a receive thread, and a main thread (which also handles the sending), which you can see below. The only thing not included below is the fect I can now call “terminate” on MessageReceiver and it will exit.

---

ORIGINAL POST
I am trying to wrap a thread around some receiving logic in python. Basically we have an app, that will have a thread in the background polling for messages, the problem I ran into is that piece that actually pulls the messages waits forever for a message. Making it impossible to terminate… I ended up wrapping the pull in another thread, but I wanted to make sure there wasn’t a better way to do it.

Original code:

class Manager:
   def __init__(self):
        receiver = MessageReceiver()
        receiver.start()
        #do other stuff sending/etc...

class MessageReceiver(Thread):
   receiver = Receiver()

   def __init__(self):
        Thread.__init__(self)


   def run(self):           
       #stop is a flag that i use to stop the thread...
       while(not stopped ):
          #can never stop because pull below blocks
          message  = receiver.pull()
          print "Message" + message   

I know the obvious locking issues exist, but is this the appropriate way to control a receive thread that waits forever for a message?

One thing I did notice was this thing eats 100% cpu while waiting for a message…
**If you need to see the stopping logic please let me know and I will post.

Answer 1

Since there was never a real answer to this question I will summarize:

1. One Main thread which handles the sending, and spawns a receiving thread
2. Block forever, exceptions are costly, so there is no need to let it time out.

In order to stop the thread I basically set the stop flag then terminate the underlying connection in a “terminate” message of the parent. In return which causes an exception in the child.

In the exception handling block I then check the ConnectionException, if the”stop” flag has been set, I gracefully stop the receiving thread, if it was an ungraceful exit, I will then notify the parent thread.

Hope this helps, this was a super annoying problem, but I am glad it is over.

class Manager:
   def __init__(self):
      self.receiver = MessageReceiver(shutdown_hook = self.shutdown_hook)
      self.receiver.start()
      #do other stuff sending/etc...
   def terminate(self):
      self.receiver.stop()
      self.receiver.close()

   def shutdown_hook(self, t_id, child):
       print '%s - Unexpected thread shutdown, handle this.. restart thread?' % str(t_id))

class MessageReceiver(Thread):


   def __init__(self,shutdown_hook = None):
     Thread.__init__(self)
     self.receiver = Receiver()
     self.shutdown_hook = shutdown_hook 


   def run(self):           
     #stop is a flag that i use to stop the thread...
     while(not stopped ):
      try:
        message  = self.receiver.pull()
        print "Message" + message   
      except ConnectionException as e:
         if stopped:
            #we are in the process of stopping
            pass
         else:
            self.shutdown_hook(self.iden, e)
            break
      finally:
            receiver.close()