Beyond Time-Average Convergence: Near-Optimal Uncoupled Online Learning
  via Clairvoyant Multiplicative Weights Update
  In this paper, we provide a novel and simple algorithm, Clairvoyant
Multiplicative Weights Updates (CMWU) for regret minimization in general games.
CMWU effectively corresponds to the standard MWU algorithm but where all
agents, when updating their mixed strategies, use the payoff profiles based on
tomorrow's behavior, i.e. the agents are clairvoyant. CMWU achieves constant
regret of $\ln(m)/\eta$ in all normal-form games with m actions and fixed
step-sizes $\eta$. Although CMWU encodes in its definition a fixed point
computation, which in principle could result in dynamics that are neither
computationally efficient nor uncoupled, we show that both of these issues can
be largely circumvented. Specifically, as long as the step-size $\eta$ is upper
bounded by $\frac{1}{(n-1)V}$, where $n$ is the number of agents and $[0,V]$ is
the payoff range, then the CMWU updates can be computed linearly fast via a
contraction map. This implementation results in an uncoupled online learning
dynamic that admits a $o (\log T)$-sparse sub-sequence where each agent
experiences at most $O(nV\log m)$ regret. This implies that the CMWU dynamics
converge with rate $O(nV \log mW( T) / T)$ to a Coarse Correlated Equilibrium
where $W(T)$ is the inverse of the function $g(t):=t\cdot 2^t$. The latter
improves on the current state-of-the-art convergence rate of uncoupled online
learning dynamics.
