Energy-aware optimization of UAV base stations placement via
  decentralized multi-agent Q-learning
  Unmanned aerial vehicles serving as aerial base stations (UAV-BSs) can be
deployed to provide wireless connectivity to ground devices in events of
increased network demand, points-of-failure in existing infrastructure, or
disasters. However, it is challenging to conserve the energy of UAVs during
prolonged coverage tasks, considering their limited on-board battery capacity.
Reinforcement learning-based (RL) approaches have been previously used to
improve energy utilization of multiple UAVs, however, a central cloud
controller is assumed to have complete knowledge of the end-devices' locations,
i.e., the controller periodically scans and sends updates for UAV
decision-making. This assumption is impractical in dynamic network environments
with UAVs serving mobile ground devices. To address this problem, we propose a
decentralized Q-learning approach, where each UAV-BS is equipped with an
autonomous agent that maximizes the connectivity of mobile ground devices while
improving its energy utilization. Experimental results show that the proposed
design significantly outperforms the centralized approaches in jointly
maximizing the number of connected ground devices and the energy utilization of
the UAV-BSs.
