I have a system that will generate messages sporadically, and I would like to only submit either zero or one message every 5 minutes. If no message is generated, nothing would be processed by the queue consumer. If a hundred identical messages are generated within 5 minutes I only want one of those to be consumed from the queue.
I am using AMQP(RabbitMQ), is there a way to accomplish this within rabbitmq or the AMQP protocol? Can I inspect a queue’s contents to ensure that I don’t insert a duplicate? It seems that queue inspection is a bad idea and not typically what should be done for a messaging system.
Without queue inspection, can this be accomplished with these tools? The only solution that comes to mind is have a second queue that takes all messages, then the consumer reads each message and puts it in an internal queue, waits for 5 minutes, and any duplicate messages that are received are discarded. After the delay, the single message is put on the “real” queue to be processed.
It seems like this might be a common situation, that a queue system could handle. Any ideas?
Handling message state is not something that AMQP is dealing with. You could try to handle the state in some process which is also present on the AMQ and proxy messages (like you wrote in your original question). Depending on your situation your maybe could use sequences or more sophisticated means of stateless duplicate detection but AMQP does not handle this specific use case.
The only thing that is guaranteed is that if multiple bindings match a message which is subsequently delivered to a queue only one message is actually delivered.