Learning Robust Scheduling with Search and Attention
  Allocating physical layer resources to users based on channel quality, buffer
size, requirements and constraints represents one of the central optimization
problems in the management of radio resources. The solution space grows
combinatorially with the cardinality of each dimension making it hard to find
optimal solutions using an exhaustive search or even classical optimization
algorithms given the stringent time requirements. This problem is even more
pronounced in MU-MIMO scheduling where the scheduler can assign multiple users
to the same time-frequency physical resources. Traditional approaches thus
resort to designing heuristics that trade optimality in favor of feasibility of
execution. In this work we treat the MU-MIMO scheduling problem as a
tree-structured combinatorial problem and, borrowing from the recent successes
of AlphaGo Zero, we investigate the feasibility of searching for the best
performing solutions using a combination of Monte Carlo Tree Search and
Reinforcement Learning. To cater to the nature of the problem at hand, like the
lack of an intrinsic ordering of the users as well as the importance of
dependencies between combinations of users, we make fundamental modifications
to the neural network architecture by introducing the self-attention mechanism.
We then demonstrate that the resulting approach is not only feasible but vastly
outperforms state-of-the-art heuristic-based scheduling approaches in the
presence of measurement uncertainties and finite buffers.
