Dynamic Resource Management for Providing QoS in Drone Delivery Systems
  Drones have been considered as an alternative means of package delivery to
reduce the delivery cost and time. Due to the battery limitations, the drones
are best suited for last-mile delivery, i.e., the delivery from the package
distribution centers (PDCs) to the customers. Since a typical delivery system
consists of multiple PDCs, each having random and time-varying demands, the
dynamic drone-to-PDC allocation would be of great importance in meeting the
demand in an efficient manner. In this paper, we study the dynamic UAV
assignment problem for a drone delivery system with the goal of providing
measurable Quality of Service (QoS) guarantees. We adopt a queueing theoretic
approach to model the customer-service nature of the problem. Furthermore, we
take a deep reinforcement learning approach to obtain a dynamic policy for the
re-allocation of the UAVs. This policy guarantees a probabilistic upper-bound
on the queue length of the packages waiting in each PDC, which is beneficial
from both the service provider's and the customers' viewpoints. We evaluate the
performance of our proposed algorithm by considering three broad arrival
classes, including Bernoulli, Time-Varying Bernoulli, and Markov-Modulated
Bernoulli arrivals. Our results show that the proposed method outperforms the
baselines, particularly in scenarios with Time-Varying and Markov-Modulated
Bernoulli arrivals, which are more representative of real-world demand
patterns. Moreover, our algorithm satisfies the QoS constraints in all the
studied scenarios while minimizing the average number of UAVs in use.
