Multi-Agent Reinforcement Learning for Network Load Balancing in Data
  Center
  This paper presents the network load balancing problem, a challenging
real-world task for multi-agent reinforcement learning (MARL) methods.
Traditional heuristic solutions like Weighted-Cost Multi-Path (WCMP) and Local
Shortest Queue (LSQ) are less flexible to the changing workload distributions
and arrival rates, with a poor balance among multiple load balancers. The
cooperative network load balancing task is formulated as a Dec-POMDP problem,
which naturally induces the MARL methods. To bridge the reality gap for
applying learning-based methods, all methods are directly trained and evaluated
on an emulation system from moderate-to large-scale. Experiments on realistic
testbeds show that the independent and "selfish" load balancing strategies are
not necessarily the globally optimal ones, while the proposed MARL solution has
a superior performance over different realistic settings. Additionally, the
potential difficulties of MARL methods for network load balancing are analysed,
which helps to draw the attention of the learning and network communities to
such challenges.
