Question

I’m writing a custom sub-class of Queue.Queue and run into a situation where I need to acquire a queue-level lock when my custom put() is being called. I’d like to reuse the existing lock Queue has (Queue.mutex), but can’t because its not an RLock.

In the source (python 2.6), it says:

# Override these methods [_put, _get, etc] to implement other queue organizations
# (e.g. stack or priority queue).
# These will only be called with appropriate locks held

But the online documentation doesn’t mention overriding them. The other Queue implementations in that module override these. So, I’m kinda inclined to believe that the _put method is package-private and isn’t really intended for use outside the Queue module.

Does anyone know how kosher it would be to use Queue._put and friends in my own subclass?

Answer 1

As far as I’m concerned, it would be completely kosher. In Python, the source is the documentation, or at least should be considered as a supplement to the published API documentation. This is largely a consequence of the enforced use of whitespace and coding conventions that emphasize clear, readable code: when you have questions that the documentation doesn’t answer, you’re supposed to be able to go to the source code and look up the answers.

In particular, details like this one are irrelevant to most clients of the Queue module (who are simply using the classes), so they wouldn’t make it into the published docs. But if you want to subclass Queue, the developers expect you to be digging deeper, so the note is there in the source code for you to find.

I will also mention that Python doesn’t really have a concept of “package-private.” Semantically, there’s no difference between a subclass which is in the same module and one which is in a different module, even one you write yourself. In fact, Python doesn’t even have a concept of “private”. Instead, it relies on a principle of responsible usage: as a developer you’re expected to be smart enough not to use internal methods when you don’t need to. Starting a member name with an underscore is simply a clue that it’s an internal method, and that you probably shouldn’t be accessing it if you are simply using the class – but if you’re subclassing it, anything is fair game.