Scheduling in Parallel Finite Buffer Systems: Optimal Decisions under
  Delayed Feedback
  Scheduling decisions in parallel queuing systems arise as a fundamental
problem, underlying the dimensioning and operation of many computing and
communication systems, such as job routing in data center clusters, multipath
communication, and Big Data systems. In essence, the scheduler maps each
arriving job to one of the possibly heterogeneous servers while aiming at an
optimization goal such as load balancing, low average delay or low loss rate.
One main difficulty in finding optimal scheduling decisions here is that the
scheduler only partially observes the impact of its decisions, e.g., through
the delayed acknowledgements of the served jobs. In this paper, we provide a
partially observable (PO) model that captures the scheduling decisions in
parallel queuing systems under limited information of delayed acknowledgements.
We present a simulation model for this PO system to find a near-optimal
scheduling policy in real-time using a scalable Monte Carlo tree search
algorithm. We numerically show that the resulting policy outperforms other
limited information scheduling strategies such as variants of
Join-the-Most-Observations and has comparable performance to full information
strategies like: Join-the-Shortest-Queue, Join-the- Shortest-Queue(d) and
Shortest-Expected-Delay. Finally, we show how our approach can optimise the
real-time parallel processing by using network data provided by Kaggle.
