Question

New to python and trying to understand multi-threading. Here’s an example from python documentation on Queue

For the heck of my life, I don’t understand how this example is working. In the worker() function, there’s an infinite loop. How does the worker know when to get out of the loop? There seems to be no breaking condition.

And what exactly is the join doing at the end? Shouldn’t I be joining the threads instead?

def worker():
    while True:
        item = q.get()
        do_work(item)
        q.task_done()

q = Queue()
for i in range(num_worker_threads):
    t = Thread(target=worker)
    t.daemon = True
    t.start()

for item in source():
    q.put(item)

q.join()       # block until all tasks are done

Also another question, When should multithreading be used and when should multiprocessing be used?

Answer 1

Yup. You’re right. worker will run forever. However since Queue only has a finite number of items, eventually worker will permanently block at q.get() (Since there will be no more items in the queue). At this point, it’s inconsequential that worker is still running. q.join() blocks until the Queue count drops to 0 (whenever the worker thread calls q.task_done, the count drops by 1). After that, the program ends. And the infinitely blocking thread dies with it’s creator.