A quick question about work balancing.
Program processing files in parallel. Lets say size of the file is an approximated measure of how long it will take to process it. All files are known in advance.
We have N nodes that can work on files. How to distribute these files so that each node will have closest to avg amount of work.
Idea is pretty trivial and I have couple of ideas, but it really seems like some classical problem with best solution already in existence.
I just don’t know what it called.
Someone knows this ?
Thanks!
EDIT:
Ok, sorry, I omitted a lot of information. I am working on MPI implementation. Standard master-slave system. One master node examines target directory, picking up files that needs to be processed, and then assign files to slaves MPI tasks so they can do their part in parallel.
Amount of slave nodes is less than 32.
Amount of target files less than 10000.
You are asking about the classic mulitprocessor scheduling problem. The wikipedia article is a good start for a basic overview of an algorithm (http://en.wikipedia.org/wiki/Multiprocessor_scheduling).