Question

I have a script that retrieves some twitter usernames and I want to get more information from twitter for those users. Because there will be a lot of users I want to process them in parallel using Resque.

In my main script I use Oauth to generate the access token but how do I pass it to workers correctly?

This is the code that starts the workers:

work_data = @messages[0...100]
Resque.enqueue(TwitterWorker, work_data, access_token)

Where access_token is a OAuth::AccessToken

When I start the worker I get an error like so:

failed: #<NoMethodError: undefined method `request' for "#<OAuth::AccessToken:0x007f8ecb9a7fe8>":String>

So I guess that the access_token param doesn’t get serialized or unserialized properly.

I could retrieve the access token in the worker, but it seems redundant and I also want them to be fast.

Answer 1

Okay, so I’ve figured it out on my own. I serialized the token before creating the job and passed the serial token as a parameter then I unserialized it the worker.

work_data = @messages[0...100]
serial_token = YAML::dump(access_token)
Resque.enqueue(TwitterWorker, work_data, serial_token)

and in my worker I just unserialized it:

def self.perfom(data, serial_token)
  token = YAML::load(serial_token)
  ....
end