Question

Using Mechanize, I need to find some way to limit requests to 1 per second(or 1 every 5 seconds, or 2 every minute, etc the point is find some way to rate limit requests).

Searching, this seems to be the way to begin to approach the issue: pre/post connect hooks. Only I don’t exactly know what to do with them or how to approach, I’m guessing from my level and research I need to do a lambda or proc that says ‘hey wait a second’, seems basic enough.

My question is basically for an example or another clue on how to do this. I tried several lambdas(and I’m at a low level of understanding in what exactly this would do):

@agent.pre_connect_hooks << lambda { |pc| sleep 1 }

but this just turns my requests to errors:

ArgumentError: wrong number of arguments (2 for 0)

Even beginning to go through the mechanize code yields little for me so far.

Any input and learning guidance appreciated.

Answer 1

Use a Proc instead :

@agent.pre_connect_hooks << Proc.new { sleep 1 }