Question

I’m processing a SQL table, but it has over 20,000 rows so it is taking forever.

So I was thinking about having a thread manager: Send 10-20 threads to process the SQL results and then after they complete send 10 more. I have a column in the DB to indicate if the table has been processed.

I’m using Sequel and no Rails.

Answer 1

Here’s how you can create 10 threads in ruby and wait for them to complete:

threads = []
0..10.times do 
  threads << Thread.new do
    processDb()
  end
end
threads.each { |t| t.join } # wait for thread to finish