I’m looking into building a distributed application and would like some advice on the best way to go about it.
My system will comprise of a central management web application where tasks will be defined, and a number of task runners which will be installed on different machines to processes those tasks.
What I need to work out is how to distribute the tasks to the clients? Should I just create a queue of tasks, and the task runners constantly poll the queue to see if a new task is ready? What would be the best way to ensure a task is only processed by one task runner? How could I also ensure tasks get evenly distributed between the task runners, so that one task runner doesn’t end up processing loads, and other only process a handfull?
The only other approach I could think of is that each task runner makes a TCP connection to the server and registers it’s interest, then when a new task is ready, the web app chooses a task runner and pushes the task down to that task runner to process?
Anybody got any other ideas? pointers? or comments?
Many thanks
Matt
A general approach to this problem would be:
Implement a set of WCF services that expose the “task running” functionality as operations;
Deploy the WCF services on several hosts/machines hosted as Windows services (they all must use the same contract);
Configure the WCF services to use the Microsoft Message Queuing binding (
netMsmqBinding), all using the same queue, in transactional mode;Configure the individual management applications to send their requests to the message queue, using the same
netMsmqBinding;Optionally, deploy the management apps/services in a load-balanced network such as an IIS Web Farm (if you want more scalability on the front-end).
This is probably the most frictionless approach as you can rely on proven tools to handle the load balancing (IIS/NLB), message security and serialization (WCF), and message delivery and transactional management (MSMQ). Most of the work is already done for you.
Of course, there’s a lot of ground to cover with respect to the actual implementation and deployment, but that might be a bit too much material to cover in a single question/answer here. This should at least point you in the right general direction.