Question

Multi-threading usually means locking critical sections, etc. So I can’t help but wonder, in a single-threaded program or a multi-threaded program but where the queue is just used in one particular thread, is there some (unnecessary) locking type of overhead?

For instance when you call put or get or qsize, etc, does it lock then do some processing then release the lock?

Answer 1

Locks are hard-coded into the Queue class. Thus, the methods put, get will use locks no matter how many threads exist in your program. The Queue is a module used to facilitate communications between threads.

Check Queue.py‘s implementation

class Queue:
    def __init__(self, maxsize=0):
        ...
        self.mutex = threading.Lock()
        self.not_empty = threading.Condition(self.mutex)
        self.not_full = threading.Condition(self.mutex)
        self.all_tasks_done = threading.Condition(self.mutex)
        self.unfinished_tasks = 0

And its put method:

def put(self, item, block=True, timeout=None):
    ...
    self.not_full.acquire()
    try:
        if self.maxsize > 0:
            ...
            elif timeout is None:
                while self._qsize() == self.maxsize:
                    self.not_full.wait()
        self._put(item)
        self.unfinished_tasks += 1
        self.not_empty.notify()
    finally:
        self.not_full.release()

I hope this answers your question.

UPDATE:

Even the method qsize uses locks:

def qsize(self):
    """Return the approximate size of the queue (not reliable!)."""
    self.mutex.acquire()
    n = self._qsize()
    self.mutex.release()
    return n

FYI, i checked Queue.py‘s implementation for Python2.7