I have a generic check that needs to be run on ca. 1000 objects. The check takes about 3 seconds. We have a server with 4 processors (and we also have other multi-processor servers in our network) so we would like to create an exe / dll to do the checking and return the results to the “master”.
Does anyone know of a framework for this, or how would one go about it in C#?
Specifically:
- What’s the best way to transfer data between the master and the worker process?
- How would the master ensure that always 4 processes are running at any one time and as soon as a worker process is finished start a new one.
- How to register that the worker is finished and append it’s results to a list?
Hope it’s clear enough but happy to clarify.
A.
Some clarifications
* There is actually no interprocess communication outside the call and return of the process e.g.
ResultObject = WorkerProcess(HeresYourDataSonDoSomethingWithIt);
* Initially one machine is a must, but thinking further down the line we will probably have some cases where we have 6000 objects to check and we would like to farm it out to multiple servers, so we would like to make the right design choice from the beginning, or at least not develop a solution for one server that would have to be completely rewritten for multiple.
Thanks!
Take a look at the builtin
ThreadPool. You can queue a “work item” for the thread pool. .NET will manage the threads, start new threads as required and make sure the “idle” threads get new work. Also, work should be equally distributed among the CPUs by the thread pool.