Joint Cluster Head Selection and Trajectory Planning in UAV-Aided IoT
  Networks by Reinforcement Learning with Sequential Model
  Employing unmanned aerial vehicles (UAVs) has attracted growing interests and
emerged as the state-of-the-art technology for data collection in
Internet-of-Things (IoT) networks. In this paper, with the objective of
minimizing the total energy consumption of the UAV-IoT system, we formulate the
problem of jointly designing the UAV's trajectory and selecting cluster heads
in the IoT network as a constrained combinatorial optimization problem which is
classified as NP-hard and challenging to solve. We propose a novel deep
reinforcement learning (DRL) with a sequential model strategy that can
effectively learn the policy represented by a sequence-to-sequence neural
network for the UAV's trajectory design in an unsupervised manner. Through
extensive simulations, the obtained results show that the proposed DRL method
can find the UAV's trajectory that requires much less energy consumption when
compared to other baseline algorithms and achieves close-to-optimal
performance. In addition, simulation results show that the trained model by our
proposed DRL algorithm has an excellent generalization ability to larger
problem sizes without the need to retrain the model.
