Question

We are developing a system for processing of different jobs by using pool of backend servers.
I am searching for an algorithm which will allow resource allocation for incoming job requests.

List or schedule of jobs is not known in advance. So resource allocation for jobs needs to be done as and when job request is received, I can’t make full plan for all jobs in advance. Also different jobs will have different priorities.To make matter worse, exact execution time of job is also not known in advance.

We have developed some tools which will provide each servers performance/load information to my scheduler. So this information can be used for scheduling purpose.

Now based on these requirements and constrains, can any one point me to any algorithm which can be useful for developing scheduling engine for such dynamic / ad-hoc jobs?

Answer 1

Unfortulately I could not find any silver bullet for my problem, no single algorithm was able to solve my resource allocation issue. So we used simple bin-packing algorithm itself with little of system behavior prediction using history data and couple of background threads doing load balancing based on current load.

so to cut story short, no algorithm available to do resource allocation in beginning itself. with help of history behavior and current load resource balancing can be done.