SREC: Proactive Self-Remedy of Energy-Constrained UAV-Based Networks via
  Deep Reinforcement Learning
  Energy-aware control for multiple unmanned aerial vehicles (UAVs) is one of
the major research interests in UAV based networking. Yet few existing works
have focused on how the network should react around the timing when the UAV
lineup is changed. In this work, we study proactive self-remedy of
energy-constrained UAV networks when one or more UAVs are short of energy and
about to quit for charging. We target at an energy-aware optimal UAV control
policy which proactively relocates the UAVs when any UAV is about to quit the
network, rather than passively dispatches the remaining UAVs after the quit.
Specifically, a deep reinforcement learning (DRL)-based self remedy approach,
named SREC-DRL, is proposed to maximize the accumulated user satisfaction
scores for a certain period within which at least one UAV will quit the
network. To handle the continuous state and action space in the problem, the
state-of-the-art algorithm of the actor-critic DRL, i.e., deep deterministic
policy gradient (DDPG), is applied with better convergence stability. Numerical
results demonstrate that compared with the passive reaction method, the
proposed SREC-DRL approach shows a $12.12\%$ gain in accumulative user
satisfaction score during the remedy period.
