Hierarchical Multi-Agent DRL-Based Framework for Joint Multi-RAT
  Assignment and Dynamic Resource Allocation in Next-Generation HetNets
  This paper considers the problem of cost-aware downlink sum-rate maximization
via joint optimal radio access technologies (RATs) assignment and power
allocation in next-generation heterogeneous wireless networks (HetNets). We
consider a future HetNet comprised of multi-RATs and serving multi-connectivity
edge devices (EDs), and we formulate the problem as mixed-integer non-linear
programming (MINP) problem. Due to the high complexity and combinatorial nature
of this problem and the difficulty to solve it using conventional methods, we
propose a hierarchical multi-agent deep reinforcement learning (DRL)-based
framework, called DeepRAT, to solve it efficiently and learn system dynamics.
In particular, the DeepRAT framework decomposes the problem into two main
stages; the RATs-EDs assignment stage, which implements a single-agent Deep Q
Network (DQN) algorithm, and the power allocation stage, which utilizes a
multi-agent Deep Deterministic Policy Gradient (DDPG) algorithm. Using
simulations, we demonstrate how the various DRL agents efficiently interact to
learn system dynamics and derive the global optimal policy. Furthermore, our
simulation results show that the proposed DeepRAT algorithm outperforms
existing state-of-the-art heuristic approaches in terms of network utility.
Finally, we quantitatively show the ability of the DeepRAT model to quickly and
dynamically adapt to abrupt changes in network dynamics, such as EDs mobility.
