Question

I have a job right now that will never quit. It involves running a data migration for a bunch of product brands.

I have a class called MigrationJob. It loops through every record in the Brands table and calls a “migrate” method on each one, which moves the data over from a different database.

include Migration

class MigrationJob

  def perform

    while true

      for brand in Brand.all

       puts "Migrating account #{brand.name}"

       brand.migrate

     end

  end

end

But see, I have the whole migration in delayed_job. This just seems wrong. Should I instead just have an individual brand’s migration in delayed_job, so that I can more easily track failures?

I would actually really like to do that. But where should I then put the code that performs the constant migrations? In a rake task?

Answer 1

When you say you have the whole migration in a delayed_job. I assume you mean that you are saying ‘handle_asynchronously’ for the perform method. If you wanted to run each individual migration as a delayed:

include Migration

 class MigrationJob

    def perform

      while true

        for brand in Brand.all

        puts "Migrating account #{brand.name}"

        brand.delay.migrate

     end
   end
 end

Now you can track which of the jobs fails, how long each takes, etc.