Given a set of processes running on a cluster of hosts, I’m designing a system that load balances the hosts through live migrations (carried out by an external service) of the processes.
The goal of the load balancing is to minimize or prevent resource starvation, a situation in which processes are not allocated the amount of resources they want to consume. There are several possible cases: (1) all hosts in the cluster are overloaded, and we want to distribute resources evenly across demanding processes. (2) We have an imbalanced cluster, we want to bring it to a balanced state as soon as possible while minimizing the cost.
What are the core factors I need to take into account in designing this system?