Queue-Learning: A Reinforcement Learning Approach for Providing Quality
  of Service
  End-to-end delay is a critical attribute of quality of service (QoS) in
application domains such as cloud computing and computer networks. This metric
is particularly important in tandem service systems, where the end-to-end
service is provided through a chain of services. Service-rate control is a
common mechanism for providing QoS guarantees in service systems. In this
paper, we introduce a reinforcement learning-based (RL-based) service-rate
controller that provides probabilistic upper-bounds on the end-to-end delay of
the system, while preventing the overuse of service resources. In order to have
a general framework, we use queueing theory to model the service systems.
However, we adopt an RL-based approach to avoid the limitations of
queueing-theoretic methods. In particular, we use Deep Deterministic Policy
Gradient (DDPG) to learn the service rates (action) as a function of the queue
lengths (state) in tandem service systems. In contrast to existing RL-based
methods that quantify their performance by the achieved overall reward, which
could be hard to interpret or even misleading, our proposed controller provides
explicit probabilistic guarantees on the end-to-end delay of the system. The
evaluations are presented for a tandem queueing system with non-exponential
inter-arrival and service times, the results of which validate our controller's
capability in meeting QoS constraints.
