Structure-aware reinforcement learning for node-overload protection in
  mobile edge computing
  Mobile Edge Computing (MEC) refers to the concept of placing computational
capability and applications at the edge of the network, providing benefits such
as reduced latency in handling client requests, reduced network congestion, and
improved performance of applications. The performance and reliability of MEC
are degraded significantly when one or several edge servers in the cluster are
overloaded. Especially when a server crashes due to the overload, it causes
service failures in MEC. In this work, an adaptive admission control policy to
prevent edge node from getting overloaded is presented. This approach is based
on a recently-proposed low complexity RL (Reinforcement Learning) algorithm
called SALMUT (Structure-Aware Learning for Multiple Thresholds), which
exploits the structure of the optimal admission control policy in multi-class
queues for an average-cost setting. We extend the framework to work for node
overload-protection problem in a discounted-cost setting. The proposed solution
is validated using several scenarios mimicking real-world deployments in two
different settings - computer simulations and a docker testbed. Our empirical
evaluations show that the total discounted cost incurred by SALMUT is similar
to state-of-the-art deep RL algorithms such as PPO (Proximal Policy
Optimization) and A2C (Advantage Actor Critic) but requires an order of
magnitude less time to train, outputs easily interpretable policy, and can be
deployed in an online manner.
