Question

I have a small app that requires some tasks to run in the background. It’s main purpose is to open a socket to another machine, to send a file over the socket (there can be N machines present) and to listen to the response on another socket. This task might take from 30 seconds to maybe a few hours and the machine is busy for this time and can’t handle any new jobs. Ideally i want to process as many tasks as possible (== non busy machines available) in parallel.

The basic application flow would be:

loop do
  # get available machines
  # fork a thread / background worker for each
  # process a wating job job
end

Can anyone recommend something simple for the described requirements that runs on windows?

Thanks!
Ben

Answer 1

As long as you are fine with “green” (non-native) threads, how about this:

require 'thread'
job_lock = Mutex.new
kid_lock = Mutex.new

$o = Mutex.new
def o(msg)
  $o.synchronize{
    t = Time.now; puts "%02d:%06.3f %s" % [ t.min, t.to_f%60, msg]
  }
end

# Simulate jobs coming into a queue
current_chores = []
CHORES = [
  "mow the cat", "wash the lawn", "order pizza",
  "answer questions on s/o", "cover cat in soothing lotion",
  "rinse suds off lawn", "pay more attention"
]
Thread.new do
  loop do
    sleep rand*5+3
    job_lock.synchronize do
      unless CHORES.empty?
        current_chores << (j=CHORES.shift)
        o "New job in the queue: '#{j}'"
      end
    end
  end
end

# A simple queue of machines to use
free_kids = %w[ Jimmy Susan ]

# The meat
loop do
  until job = job_lock.synchronize{ current_chores.shift }
    o "Waiting for a job..."
    sleep 1 # Look for new jobs every second
  end
  until kid = kid_lock.synchronize{ free_kids.shift }
    o "Waiting for a free child to do my bidding..."
    sleep 2 # Look for new kids every 2 seconds
  end
  Thread.new do
    o "#{kid} is now performing '#{job}'"
    sleep rand*10+10 # Simulate long process
    o "#{kid} FINISHED '#{job}'"
    kid_lock.synchronize{ free_kids << kid }
  end
end

#=> 08:52.604 Waiting for a job...
#=> 08:53.604 Waiting for a job...
#=> 08:54.604 Waiting for a job...
#=> 08:55.604 Waiting for a job...
#=> 08:56.605 Waiting for a job...
#=> 08:57.327 New job in the queue: 'mow the cat'
#=> 08:57.605 Waiting for a job...
#=> 08:57.606 Jimmy is now performing 'mow the cat'
#=> 08:58.606 Waiting for a job...
#=> 08:59.606 Waiting for a job...
#=> 09:00.606 Waiting for a job...
#=> 09:01.606 Waiting for a job...
#=> 09:01.626 New job in the queue: 'wash the lawn'
#=> 09:02.606 Waiting for a job...
#=> 09:02.607 Susan is now performing 'wash the lawn'
#=> 09:03.607 Waiting for a job...
#=> 09:04.607 Waiting for a job...
#=> 09:05.607 Waiting for a job...
#=> 09:06.607 Waiting for a job...
#=> 09:07.431 New job in the queue: 'order pizza'
#=> 09:07.607 Waiting for a free child to do my bidding...
#=> 09:09.607 Waiting for a free child to do my bidding...
#=> 09:11.608 Waiting for a free child to do my bidding...
#=> 09:13.373 Jimmy FINISHED 'mow the cat'
#=> 09:13.609 Waiting for a job...
#=> 09:13.609 Jimmy is now performing 'order pizza'
#=> 09:13.930 New job in the queue: 'answer questions on s/o'
#=> 09:14.609 Waiting for a free child to do my bidding...
#=> 09:16.609 Waiting for a free child to do my bidding...
#=> 09:16.671 Susan FINISHED 'wash the lawn'
#=> 09:18.609 Waiting for a job...
#=> 09:18.610 Susan is now performing 'answer questions on s/o'
#=> 09:19.610 Waiting for a job...
#=> 09:19.878 New job in the queue: 'cover cat in soothing lotion'
#=> 09:20.611 Waiting for a free child to do my bidding...
#=> 09:22.611 Waiting for a free child to do my bidding...
#=> 09:24.196 New job in the queue: 'rinse suds off lawn'
#=> 09:24.611 Waiting for a free child to do my bidding...
#=> 09:26.612 Waiting for a free child to do my bidding...
#=> 09:28.612 Waiting for a free child to do my bidding...
#=> 09:28.677 New job in the queue: 'pay more attention'
#=> 09:29.878 Jimmy FINISHED 'order pizza'
#=> 09:30.263 Susan FINISHED 'answer questions on s/o'
#=> 09:30.613 Waiting for a free child to do my bidding...
#=> 09:30.614 Susan is now performing 'rinse suds off lawn'
#=> 09:30.615 Jimmy is now performing 'cover cat in soothing lotion'
#=> 09:32.614 Waiting for a free child to do my bidding...
#=> 09:34.614 Waiting for a free child to do my bidding...
#=> 09:36.614 Waiting for a free child to do my bidding...
#=> 09:38.614 Waiting for a free child to do my bidding...
#=> 09:40.614 Waiting for a free child to do my bidding...
#=> 09:42.614 Waiting for a free child to do my bidding...
#=> 09:42.764 Jimmy FINISHED 'cover cat in soothing lotion'
#=> 09:44.614 Waiting for a job...
#=> 09:44.615 Jimmy is now performing 'pay more attention'
#=> 09:45.615 Waiting for a job...
#=> 09:46.615 Waiting for a job...
#=> 09:47.615 Waiting for a job...
#=> 09:47.862 Susan FINISHED 'rinse suds off lawn'
#=> 09:48.615 Waiting for a job...
#=> 09:49.615 Waiting for a job...
#=> 09:50.615 Waiting for a job...
#=> 09:51.616 Waiting for a job...
#=> 09:52.617 Waiting for a job...
#=> 09:53.617 Waiting for a job...
#=> 09:54.618 Waiting for a job...
#=> 09:55.619 Waiting for a job...
#=> 09:56.619 Waiting for a job...
#=> 09:57.620 Waiting for a job...
#=> 09:58.620 Waiting for a job...
#=> 09:59.620 Waiting for a job...
#=> 10:00.531 Jimmy FINISHED 'pay more attention'
#=> 10:00.621 Waiting for a job...