Learning based Age of Information Minimization in UAV-relayed IoT
  Networks
  Unmanned Aerial Vehicles (UAVs) are used as aerial base-stations to relay
time-sensitive packets from IoT devices to the nearby terrestrial base-station
(TBS). Scheduling of packets in such UAV-relayed IoT-networks to ensure fresh
(or up-to-date) IoT devices' packets at the TBS is a challenging problem as it
involves two simultaneous steps of (i) sampling of packets generated at IoT
devices by the UAVs [hop-1] and (ii) updating of sampled packets from UAVs to
the TBS [hop-2]. To address this, we propose Age-of-Information (AoI)
scheduling algorithms for two-hop UAV-relayed IoT-networks. First, we propose a
low-complexity AoI scheduler, termed, MAF-MAD that employs Maximum AoI First
(MAF) policy for sampling of IoT devices at UAV (hop-1) and Maximum AoI
Difference (MAD) policy for updating sampled packets from UAV to the TBS
(hop-2). We prove that MAF-MAD is the optimal AoI scheduler under ideal
conditions (lossless wireless channels and generate-at-will traffic-generation
at IoT devices). On the contrary, for general conditions (lossy channel
conditions and varying periodic traffic-generation at IoT devices), a deep
reinforcement learning algorithm, namely, Proximal Policy Optimization
(PPO)-based scheduler is proposed. Simulation results show that the proposed
PPO-based scheduler outperforms other schedulers like MAF-MAD, MAF, and
round-robin in all considered general scenarios.
