On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems
  We consider nonconvex-concave minimax problems, $\min_{\mathbf{x}}
\max_{\mathbf{y} \in \mathcal{Y}} f(\mathbf{x}, \mathbf{y})$, where $f$ is
nonconvex in $\mathbf{x}$ but concave in $\mathbf{y}$ and $\mathcal{Y}$ is a
convex and bounded set. One of the most popular algorithms for solving this
problem is the celebrated gradient descent ascent (GDA) algorithm, which has
been widely used in machine learning, control theory and economics. Despite the
extensive convergence results for the convex-concave setting, GDA with equal
stepsize can converge to limit cycles or even diverge in a general setting. In
this paper, we present the complexity results on two-time-scale GDA for solving
nonconvex-concave minimax problems, showing that the algorithm can find a
stationary point of the function $\Phi(\cdot) := \max_{\mathbf{y} \in
\mathcal{Y}} f(\cdot, \mathbf{y})$ efficiently. To the best our knowledge, this
is the first nonasymptotic analysis for two-time-scale GDA in this setting,
shedding light on its superior practical performance in training generative
adversarial networks (GANs) and other real applications.
