First off, some basic information about my project: I have a website with about 1000 visitors a day, built with Symfony 2. For some tasks I’m thinking about implementing a message queue, Rabbit MQ would be my first choice based on some internet reviews.
But I think I’m going to have a lot of different task types (in the first stage about 10 tasks), and they aren’t going to be run that very much. For instance: I want a message queue for sending e-mails: I want these to be sent immediately, so creating a cron job is not really acceptable, but my website will maybe send 5 e-mails a day.
Now I’m wondering if it’s better to create message queues for every task type. In which case I will have to run 10 concurrent consumers. Or create one ‘generic’ queue, which just executes a Symfony2 Command with the parameters in the message.
I think the best and most “pure” solution would be the first (10 queues), but I’m worried it’s a bit ‘overdone’…
So does anyone have any experience with this is willing to help me?
I would recommend creating different queues for each unique task that you need to be executed, and assign priorities to the queues so that the most important actions are executed first even if they are added later on with other messages in the queue.
In your queue processor you can execute different commands using the message body as parameters.