Question

I have a simple stack with Ruby on Rails and resque. I enqueue jobs in a normal way and I have a pool of workers performing. Nothing crazy.

My problem is that if I leave the workers running for long enought they will stop to have visibility on the app’s models, and every call to such a method will result in an undefined_method.

This is very weird since it could be working perfectly for days and then suddenly it starts failing. Restarting the worker fixes the problem, but it usually come back after a while.

I have no clue what could be going on, so any pointers would be greatly appreciated.

Answer 1

Resque workers fork new processes to do the work. It’s possible that the classes for your models are not loaded in the forked child process. It’s also possible that there is a namespace collision because of the order that classes are loaded.

If you are changing class files in development without restarting your resque workers, I suspect it’s not reloading your classes correctly.

To make sure that your classes are loaded pre-fork, reference the classes in the resque setup task. The classes that are loaded pre-fork will be copied to the child process. Below, I put them in an array to force them to load. This is also faster, since each child process will already have the classes loaded. Also you should re-establish your ActiveRecord connection in an after_fork block if you’re using AR.

lib/tasks/resque.rake:

namespace :resque do
  task :setup => :environment do
    [User, Monkey, Banana] # force these classes to load

    Resque.after_fork { ActiveRecord::Base.establish_connection }
  end
end