Process A is calculating values for objects a1, a2, a3 etc. and is sending results to the middleware queue (RabbitMQ). Consumers read the queue and process these results further.
Periodically process A has to send a snapshot of these values, so consumers could do some other calculations. Values for these objects might change independently. The queue might look like this a1, a1, a2, a1, a2, a2, a3… Consumers process each item in the queue. The snapshot has to contain all objects and consumers will process this message for all objects in one go.
So the requirement is to have a queue like this: a1, a1, a3, a2, a2, [snapshot, a1, a2, a3], a3, a1 … The problem is that these items are of different types: one type for objects like a1, a2 and other for a snapshot. This means that they should be processed in a diferent queues, but in that case there is a race condition: consumers might process objects before processing a snapshot.
Is there any pattern to solve this (quite common) problem? We are using RabbitMQ for message queueing.
Use a single queue per consumer and a content-type prefix to indicate the type of message being delivered. In think there may be a ‘type’ attribute in AMQP messages, but I’ve never used it, so I don’t know if it’s applicable to this problem.