Convergence Rates of Two-Time-Scale Gradient Descent-Ascent Dynamics for
  Solving Nonconvex Min-Max Problems
  There are much recent interests in solving noncovnex min-max optimization
problems due to its broad applications in many areas including machine
learning, networked resource allocations, and distributed optimization.
Perhaps, the most popular first-order method in solving min-max optimization is
the so-called simultaneous (or single-loop) gradient descent-ascent algorithm
due to its simplicity in implementation. However, theoretical guarantees on the
convergence of this algorithm is very sparse since it can diverge even in a
simple bilinear problem.
  In this paper, our focus is to characterize the finite-time performance (or
convergence rates) of the continuous-time variant of simultaneous gradient
descent-ascent algorithm. In particular, we derive the rates of convergence of
this method under a number of different conditions on the underlying objective
function, namely, two-sided Polyak-L ojasiewicz (PL), one-sided PL,
nonconvex-strongly concave, and strongly convex-nonconcave conditions. Our
convergence results improve the ones in prior works under the same conditions
of objective functions. The key idea in our analysis is to use the classic
singular perturbation theory and coupling Lyapunov functions to address the
time-scale difference and interactions between the gradient descent and ascent
dynamics. Our results on the behavior of continuous-time algorithm may be used
to enhance the convergence properties of its discrete-time counterpart.
