Convergence Analysis of Gradient-Based Learning with Non-Uniform
  Learning Rates in Non-Cooperative Multi-Agent Settings
  Considering a class of gradient-based multi-agent learning algorithms in
non-cooperative settings, we provide local convergence guarantees to a
neighborhood of a stable local Nash equilibrium. In particular, we consider
continuous games where agents learn in (i) deterministic settings with oracle
access to their gradient and (ii) stochastic settings with an unbiased
estimator of their gradient. Utilizing the minimum and maximum singular values
of the game Jacobian, we provide finite-time convergence guarantees in the
deterministic case. On the other hand, in the stochastic case, we provide
concentration bounds guaranteeing that with high probability agents will
converge to a neighborhood of a stable local Nash equilibrium in finite time.
Different than other works in this vein, we also study the effects of
non-uniform learning rates on the learning dynamics and convergence rates. We
find that much like preconditioning in optimization, non-uniform learning rates
cause a distortion in the vector field which can, in turn, change the rate of
convergence and the shape of the region of attraction. The analysis is
supported by numerical examples that illustrate different aspects of the
theory. We conclude with discussion of the results and open questions.
