Learning and Information in Stochastic Networks and Queues
  We review the role of information and learning in the stability and
optimization of queueing systems. In recent years, techniques from supervised
learning, bandit learning and reinforcement learning have been applied to
queueing systems supported by increasing role of information in decision
making. We present observations and new results that help rationalize the
application of these areas to queueing systems.
  We prove that the MaxWeight and BackPressure policies are an application of
Blackwell's Approachability Theorem. This connects queueing theoretic results
with adversarial learning. We then discuss the requirements of statistical
learning for service parameter estimation. As an example, we show how queue
size regret can be bounded when applying a perceptron algorithm to classify
service. Next, we discuss the role of state information in improved decision
making. Here we contrast the roles of epistemic information (information on
uncertain parameters) and aleatoric information (information on an uncertain
state). Finally we review recent advances in the theory of reinforcement
learning and queueing, as well as, provide discussion on current research
challenges.
