Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled
  Wireless Networks: A Tutorial
  Deep Reinforcement Learning (DRL) has recently witnessed significant advances
that have led to multiple successes in solving sequential decision-making
problems in various domains, particularly in wireless communications. The
future sixth-generation (6G) networks are expected to provide scalable,
low-latency, ultra-reliable services empowered by the application of
data-driven Artificial Intelligence (AI). The key enabling technologies of
future 6G networks, such as intelligent meta-surfaces, aerial networks, and AI
at the edge, involve more than one agent which motivates the importance of
multi-agent learning techniques. Furthermore, cooperation is central to
establishing self-organizing, self-sustaining, and decentralized networks. In
this context, this tutorial focuses on the role of DRL with an emphasis on deep
Multi-Agent Reinforcement Learning (MARL) for AI-enabled 6G networks. The first
part of this paper will present a clear overview of the mathematical frameworks
for single-agent RL and MARL. The main idea of this work is to motivate the
application of RL beyond the model-free perspective which was extensively
adopted in recent years. Thus, we provide a selective description of RL
algorithms such as Model-Based RL (MBRL) and cooperative MARL and we highlight
their potential applications in 6G wireless networks. Finally, we overview the
state-of-the-art of MARL in fields such as Mobile Edge Computing (MEC),
Unmanned Aerial Vehicles (UAV) networks, and cell-free massive MIMO, and
identify promising future research directions. We expect this tutorial to
stimulate more research endeavors to build scalable and decentralized systems
based on MARL.
