Efficient Methods for Structured Nonconvex-Nonconcave Min-Max
  Optimization
  The use of min-max optimization in adversarial training of deep neural
network classifiers and training of generative adversarial networks has
motivated the study of nonconvex-nonconcave optimization objectives, which
frequently arise in these applications. Unfortunately, recent results have
established that even approximate first-order stationary points of such
objectives are intractable, even under smoothness conditions, motivating the
study of min-max objectives with additional structure. We introduce a new class
of structured nonconvex-nonconcave min-max optimization problems, proposing a
generalization of the extragradient algorithm which provably converges to a
stationary point. The algorithm applies not only to Euclidean spaces, but also
to general $\ell_p$-normed finite-dimensional real vector spaces. We also
discuss its stability under stochastic oracles and provide bounds on its sample
complexity. Our iteration complexity and sample complexity bounds either match
or improve the best known bounds for the same or less general
nonconvex-nonconcave settings, such as those that satisfy variational coherence
or in which a weak solution to the associated variational inequality problem is
assumed to exist.
