Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep
  Reinforcement Learning Approach
  In this paper, we design a navigation policy for multiple unmanned aerial
vehicles (UAVs) where mobile base stations (BSs) are deployed to improve the
data freshness and connectivity to the Internet of Things (IoT) devices. First,
we formulate an energy-efficient trajectory optimization problem in which the
objective is to maximize the energy efficiency by optimizing the UAV-BS
trajectory policy. We also incorporate different contextual information such as
energy and age of information (AoI) constraints to ensure the data freshness at
the ground BS. Second, we propose an agile deep reinforcement learning with
experience replay model to solve the formulated problem concerning the
contextual constraints for the UAV-BS navigation. Moreover, the proposed
approach is well-suited for solving the problem, since the state space of the
problem is extremely large and finding the best trajectory policy with useful
contextual features is too complex for the UAV-BSs. By applying the proposed
trained model, an effective real-time trajectory policy for the UAV-BSs
captures the observable network states over time. Finally, the simulation
results illustrate the proposed approach is 3.6% and 3.13% more energy
efficient than those of the greedy and baseline deep Q Network (DQN)
approaches.
