I have a project which needs to make a tcp connection to an external source. Each worker thread will be sending messages to this external service.
I’m wondering how I can do this without having a connection be brought up and torn down for every request. I’m pretty sure the pymongo module does something similar but I can’t find any documentation on it. Would it be possible to set up some kind of thread-safe queue and have a separate thread consume that queue? I understand I could probably use gearman for this, but I’d like to avoid having another moving part in the system.
uWSGI has a thread-safe process-shared queueing system (http://projects.unbit.it/uwsgi/wiki/QueueFramework) but are you sure using simple python threading.Queue class is not enough ?