Last-Iterate Convergence of Saddle Point Optimizers via High-Resolution
  Differential Equations
  Several widely-used first-order saddle point optimization methods yield an
identical continuous-time ordinary differential equation (ODE) to that of the
Gradient Descent Ascent (GDA) method when derived naively. However, their
convergence properties are very different even on simple bilinear games.
  We use a technique from fluid dynamics called High-Resolution Differential
Equations (HRDEs) to design ODEs of several saddle point optimization methods.
On bilinear games, the convergence properties of the derived HRDEs correspond
to that of the starting discrete methods. Using these techniques, we show that
the HRDE of Optimistic Gradient Descent Ascent (OGDA) has last-iterate
convergence for general monotone variational inequalities. To our knowledge,
this is the first continuous-time dynamics shown to converge for such a general
setting. Moreover, we provide the rates for the best-iterate convergence of the
OGDA method, relying solely on the first-order smoothness of the monotone
operator.
