I have a request queue and a response queue. I would like to take the message out of the request queue and put it into the response queue. I would like to be able to identify that the same message that was taken out of the request queue has been put into the response queue.
I am using Boto on GAE.
Firstly, I assumed that the message id would be constant. So I read the message from request queue, added the same message object to the response queue and queried the id – SQS had generated a new one.
Secondly, the body of my message is a json object so I added a custom field ‘messageId’ and generated my own id to store with the queue. But then I saw that there is a limit of reading 10 messages at a time from the queue. In addition, it is not guaranteed that the message will be returned because of the distributed nature of SQS.
Your message will be delivered, just not necessarily on any given request to fetch it, as you’ll be speaking to a random subset of the backend servers. There is no way in SQS to say, “I know there are 3 messages for me, give them to me right now.” The best you can do is poll until you eventually receive them. SQS is pretty neat, but its not the right choice for everything, have a look at RabbitMQ if you want something with less latency, more predictability, and ordered messages. Only problem with Rabbit is now scale is your problem.