Simple Uncoupled No-Regret Learning Dynamics for Extensive-Form
  Correlated Equilibrium
  The existence of simple uncoupled no-regret learning dynamics that converge
to correlated equilibria in normal-form games is a celebrated result in the
theory of multi-agent systems. Specifically, it has been known for more than 20
years that when all players seek to minimize their internal regret in a
repeated normal-form game, the empirical frequency of play converges to a
normal-form correlated equilibrium. Extensive-form games generalize normal-form
games by modeling both sequential and simultaneous moves, as well as imperfect
information. Because of the sequential nature and presence of private
information in the game, correlation in extensive-form games possesses
significantly different properties than its counterpart in normal-form games,
many of which are still open research directions. Extensive-form correlated
equilibrium (EFCE) has been proposed as the natural extensive-form counterpart
to the classical notion of correlated equilibrium in normal-form games.
Compared to the latter, the constraints that define the set of EFCEs are
significantly more complex, as the correlation device must keep into account
the evolution of beliefs of each player as they make observations throughout
the game. Due to that significant added complexity, the existence of uncoupled
learning dynamics leading to an EFCE has remained a challenging open research
question for a long time. In this article, we settle that question by giving
the first uncoupled no-regret dynamics that converge to the set of EFCEs in
n-player general-sum extensive-form games with perfect recall. We show that
each iterate can be computed in time polynomial in the size of the game tree,
and that, when all players play repeatedly according to our learning dynamics,
the empirical frequency of play is proven to be a O(T^-0.5)-approximate EFCE
with high probability after T game repetitions, and an EFCE almost surely in
the limit.
