Question

I need different messages to be sent to the queue – each by its personal schedule. So I have message list and related interval to resend each one. I use rabbitMQ/pika and apscheduler.
According to numerous examples, I created the simplest BlockingConnection/channel/queue. When immediately after that I try to push messages – everything works fine, I can see in rabbitmq web-interface that all the messages become in the queue. Here is the piece of code that works:

    self.cr = Queue('DIRECT_C_QUEUE', True, ex_type='direct')
    for i in range(1,10000):
        self.cr.channel.basic_publish(exchange='', routing_key='DIRECT_C_QUEUE', body='hello_world')

But if I try to push messages (in exactly same way) via apscheduler callback function – only few (about 1-10) messages appear in the queue (but callbacks are fired all the time and there are no any exception when publishing message!).

Finally I begin to receive such warnings:

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pika/connection.py:642: UserWarning: Pika: Write buffer exceeded warning threshold at 1125 bytes and an estimated 43 frames behind
warn(message % (self.outbound_buffer.size, est_frames_behind))

and still no new messages in the queue.

I am new in python, any help is much appreciate.

Answer 1

I found source of the problem:
apscheduler runs basic_publish calls in separate thread, and pika doesn’t recommend share connections between threads – http://pika.github.com/faq.html

So I had choice either create new connection each time, or put new messages in some queue and publish them from the main thread (where connection was created).