Question

I’ve just switched to using Unicorn on Heroku. I’m also going to switch to resque from delayed_job and use the setup described at http://bugsplat.info/2011-11-27-concurrency-on-heroku-cedar.html

What I don’t understand from this is how config/unicorn.rb:

worker_processes 3
timeout 30

@resque_pid = nil

before_fork do |server, worker|
  @resque_pid ||= spawn("bundle exec rake " + \
  "resque:work QUEUES=scrape,geocode,distance,mailer")
end

translates into:

“This will actually result in six processes in each web dyno: 1 unicorn master, 3 unicorn web workers, 1 resque worker, 1 resque child worker when it actually is processing a job”

How many workers will actually process background jobs? 1 or 2?

Lets say I wanted to increase the number of resque workers – what would I change?

Answer 1

I think if you run that block, you have your unicorn master already running, plus 3 web workers that you specify at the top of the file, and then the block below launches one Resque worker if it’s not already started.

I’m guessing that Resque launches a child worker by itself when it actually performs work.

It would appear that if you wanted another Resque worker, you could just do

worker_processes 3
timeout 30

@resque_pid = nil
@resque_pid2 = nil

before_fork do |server, worker|
  @resque_pid ||= spawn("bundle exec rake " + \
  "resque:work QUEUES=scrape,geocode,distance,mailer")
  @resque_pid2 ||= spawn("bundle exec rake " + \
  "resque:work QUEUES=scrape,geocode,distance,mailer")
end

In my experience with Resque, it’s as simple as launching another process as specified above. The only uncertainty I have is with Heroku and how it chooses to deal with giving you more workers.