AI-based Resource Allocation: Reinforcement Learning for Adaptive
  Auto-scaling in Serverless Environments
  Serverless computing has emerged as a compelling new paradigm of cloud
computing models in recent years. It promises the user services at large scale
and low cost while eliminating the need for infrastructure management. On cloud
provider side, flexible resource management is required to meet fluctuating
demand. It can be enabled through automated provisioning and deprovisioning of
resources. A common approach among both commercial and open source serverless
computing platforms is workload-based auto-scaling, where a designated
algorithm scales instances according to the number of incoming requests. In the
recently evolving serverless framework Knative a request-based policy is
proposed, where the algorithm scales resources by a configured maximum number
of requests that can be processed in parallel per instance, the so-called
concurrency. As we show in a baseline experiment, this predefined concurrency
level can strongly influence the performance of a serverless application.
However, identifying the concurrency configuration that yields the highest
possible quality of service is a challenging task due to various factors, e.g.
varying workload and complex infrastructure characteristics, influencing
throughput and latency. While there has been considerable research into
intelligent techniques for optimizing auto-scaling for virtual machine
provisioning, this topic has not yet been discussed in the area of serverless
computing. For this reason, we investigate the applicability of a reinforcement
learning approach, which has been proven on dynamic virtual machine
provisioning, to request-based auto-scaling in a serverless framework. Our
results show that within a limited number of iterations our proposed model
learns an effective scaling policy per workload, improving the performance
compared to the default auto-scaling configuration.
