Reinforcement Learning-based Admission Control in Delay-sensitive
  Service Systems
  Ensuring quality of service (QoS) guarantees in service systems is a
challenging task, particularly when the system is composed of more fine-grained
services, such as service function chains. An important QoS metric in service
systems is the end-to-end delay, which becomes even more important in
delay-sensitive applications, where the jobs must be completed within a time
deadline. Admission control is one way of providing end-to-end delay guarantee,
where the controller accepts a job only if it has a high probability of meeting
the deadline. In this paper, we propose a reinforcement learning-based
admission controller that guarantees a probabilistic upper-bound on the
end-to-end delay of the service system, while minimizes the probability of
unnecessary rejections. Our controller only uses the queue length information
of the network and requires no knowledge about the network topology or system
parameters. Since long-term performance metrics are of great importance in
service systems, we take an average-reward reinforcement learning approach,
which is well suited to infinite horizon problems. Our evaluations verify that
the proposed RL-based admission controller is capable of providing
probabilistic bounds on the end-to-end delay of the network, without using
system model information.
