A Convergent and Dimension-Independent First-Order Algorithm for Min-Max
  Optimization
  Motivated by the recent work of Mangoubi and Vishnoi (STOC 2021), we propose
a variant of the min-max optimization framework where the max-player is
constrained to update the maximization variable in a greedy manner until it
reaches a *first-order* stationary point. We present an algorithm that provably
converges to an approximate local equilibrium for our framework from any
initialization and for nonconvex-nonconcave loss functions. Compared to the
second-order algorithm of Mangoubi and Vishnoi, whose iteration bound is
polynomial in the dimension, our algorithm is first-order and its iteration
bound is independent of dimension. We empirically evaluate our algorithm on
challenging nonconvex-nonconcave test-functions and loss functions that arise
in GAN training. Our algorithm converges on these test functions and, when used
to train GANs on synthetic and real-world datasets, trains stably and avoids
mode collapse.
