I plan on running our web app on Heroku. I am looking for a gem to handle a few background jobs. i.e. sending emails, to call a few methods which submit files to an encoding service via an API, etc.
A few that have, so far, come to mind are resque and delayed_job. I hear good things about resque and it also seems to be the more popular gem in its category. Ryan Bates has done an excellent screen cast on delayed_job. However, I hear delayed_job has had a few problems. i.e. not very stable in some areas. So I hear.
Heroku offers Redis-to-Go. They have a free plan which offers 5mb. If I go with resque, is this 5mb plan enough to handle background jobs? I don’t want to end up spending more just for background jobs.
Just concerned that if I went with resque, I would need another db just to run background jobs. If I was using Redis for something else, then perhaps it would be worth it. Is it worth having another db just to handle background jobs?
Should I consider alternative gems? If so which ones?
Both delayed_job and resque work fairly well. resque should scale better as the volume of background requests increases.
resque’s use of redis should be limited to the task request. Large data objects that are needed by the background tasks should be stored somewhere other than the background worker queue. For example, the files being sent to a background worker to be encoded should be stored in AWS S3 or some other persistent store, not the redis queue used by resque.
When using delayed_job or resque, you will need to run background workers which cost money. You might want to look at an autoscaling solution for dynamically starting and stopping background workers as needed.
See http://s831.us/h3pKE6 as an example.