Multi-Agent Deep Reinforcement Learning For Optimising Energy Efficiency
  of Fixed-Wing UAV Cellular Access Points
  Unmanned Aerial Vehicles (UAVs) promise to become an intrinsic part of next
generation communications, as they can be deployed to provide wireless
connectivity to ground users to supplement existing terrestrial networks. The
majority of the existing research into the use of UAV access points for
cellular coverage considers rotary-wing UAV designs (i.e. quadcopters).
However, we expect fixed-wing UAVs to be more appropriate for connectivity
purposes in scenarios where long flight times are necessary (such as for rural
coverage), as fixed-wing UAVs rely on a more energy-efficient form of flight
when compared to the rotary-wing design. As fixed-wing UAVs are typically
incapable of hovering in place, their deployment optimisation involves
optimising their individual flight trajectories in a way that allows them to
deliver high quality service to the ground users in an energy-efficient manner.
In this paper, we propose a multi-agent deep reinforcement learning approach to
optimise the energy efficiency of fixed-wing UAV cellular access points while
still allowing them to deliver high-quality service to users on the ground. In
our decentralized approach, each UAV is equipped with a Dueling Deep Q-Network
(DDQN) agent which can adjust the 3D trajectory of the UAV over a series of
timesteps. By coordinating with their neighbours, the UAVs adjust their
individual flight trajectories in a manner that optimises the total system
energy efficiency. We benchmark the performance of our approach against a
series of heuristic trajectory planning strategies, and demonstrate that our
method can improve the system energy efficiency by as much as 70%.
