Decentralized Learning in Online Queuing Systems
  Motivated by packet routing in computer networks, online queuing systems are
composed of queues receiving packets at different rates. Repeatedly, they send
packets to servers, each of them treating only at most one packet at a time. In
the centralized case, the number of accumulated packets remains bounded (i.e.,
the system is \textit{stable}) as long as the ratio between service rates and
arrival rates is larger than $1$. In the decentralized case, individual
no-regret strategies ensures stability when this ratio is larger than $2$. Yet,
myopically minimizing regret disregards the long term effects due to the
carryover of packets to further rounds. On the other hand, minimizing long term
costs leads to stable Nash equilibria as soon as the ratio exceeds
$\frac{e}{e-1}$. Stability with decentralized learning strategies with a ratio
below $2$ was a major remaining question. We first argue that for ratios up to
$2$, cooperation is required for stability of learning strategies, as selfish
minimization of policy regret, a \textit{patient} notion of regret, might
indeed still be unstable in this case. We therefore consider cooperative queues
and propose the first learning decentralized algorithm guaranteeing stability
of the system as long as the ratio of rates is larger than $1$, thus reaching
performances comparable to centralized strategies.
