Faster Single-loop Algorithms for Minimax Optimization without Strong
  Concavity
  Gradient descent ascent (GDA), the simplest single-loop algorithm for
nonconvex minimax optimization, is widely used in practical applications such
as generative adversarial networks (GANs) and adversarial training. Albeit its
desirable simplicity, recent work shows inferior convergence rates of GDA in
theory even assuming strong concavity of the objective on one side. This paper
establishes new convergence results for two alternative single-loop algorithms
-- alternating GDA and smoothed GDA -- under the mild assumption that the
objective satisfies the Polyak-Lojasiewicz (PL) condition about one variable.
We prove that, to find an $\epsilon$-stationary point, (i) alternating GDA and
its stochastic variant (without mini batch) respectively require $O(\kappa^{2}
\epsilon^{-2})$ and $O(\kappa^{4} \epsilon^{-4})$ iterations, while (ii)
smoothed GDA and its stochastic variant (without mini batch) respectively
require $O(\kappa \epsilon^{-2})$ and $O(\kappa^{2} \epsilon^{-4})$ iterations.
The latter greatly improves over the vanilla GDA and gives the hitherto best
known complexity results among single-loop algorithms under similar settings.
We further showcase the empirical efficiency of these algorithms in training
GANs and robust nonlinear regression.
