Question

I have code that sends a message into a MessageQueue.

_queue.Send(new Message(myData));

This queue is on the local machine and the threads that Receive() from it are in the same process.

What happens if the messages are inserted faster than they are extracted? Will Send() block?
Is there a way for me to know if the MessageQueue is full before sending into it more events?
(I would prefer at this point just to log myData and not send the event).

Thanks,
Sela.

Answer 1

Short answer: Do the simple thing and don’t limit on send.

Long answer:
The message queue will only really get full when the disk it’s allocated to save to is out of space – which is the same time that your logging will be out of space. The message queue is very good at holding data you’re not ready to process. Don’t throttle on send. If you’re concerned about system management and disk space then you might prefer to rely on Window’s excellent system monitoring facilities and disk space usage threshold alerts. You don’t need to reinvent this for your application.

That is unless you’re running the queue in memory only mode which may not be necessary. If you can’t process the messages fast enough then you definitely have enough time to let the queue manager persist the messages to disk. You should only consider running the queue in memory only mode if you’re going to scale to many consumer processes on many servers and the disk IO on the queue manager becomes the bottleneck. One process on the same machine is very far away from that scenario. Let the queue manager do what it does best. Don’t optimise prematurely.

If you implement a specified quality of service like X messages per second and bill your customers more for processing a higher quality of service then throttle at the receiving end. I’ve done this successfully using a semaphore initialised with a resource limit equal to the number of messages to consume per second. Each consumer thread took a snapshot of the message start time, processed 1 message and then waited for the end of second before giving up the semaphore. That way the thread pool could grow to accomodate the quality of service if messages took more than 1 second to process but would not exceed the quality of service.

Good luck!