Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT
  Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement
  Learning Approach
  This paper investigates a master unmanned aerial vehicle (MUAV)-powered
Internet of Things (IoT) network, in which we propose using a rechargeable
auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to
enhance the communication signals from the MUAV and also leverage the MUAV as a
recharging power source. Under the proposed model, we investigate the optimal
collaboration strategy of these energy-limited UAVs to maximize the accumulated
throughput of the IoT network. Depending on whether there is charging between
the two UAVs, two optimization problems are formulated. To solve them, two
multi-agent deep reinforcement learning (DRL) approaches are proposed, which
are centralized training multi-agent deep deterministic policy gradient
(CT-MADDPG) and multi-agent deep deterministic policy option critic (MADDPOC).
It is shown that the CT-MADDPG can greatly reduce the requirement on the
computing capability of the UAV hardware, and the proposed MADDPOC is able to
support low-level multi-agent cooperative learning in the continuous action
domains, which has great advantages over the existing option-based hierarchical
DRL that only support single-agent learning and discrete actions.
