LEAD: Least-Action Dynamics for Min-Max Optimization
  Adversarial formulations such as generative adversarial networks (GANs) have
rekindled interest in two-player min-max games. A central obstacle in the
optimization of such games is the rotational dynamics that hinder their
convergence. Existing methods typically employ intuitive, carefully
hand-designed mechanisms for controlling such rotations. In this paper, we take
a novel approach to address this issue by casting min-max optimization as a
physical system. We leverage tools from physics to introduce LEAD (Least-Action
Dynamics), a second-order optimizer for min-max games. Next, using Lyapunov
stability theory and spectral analysis, we study LEAD's convergence properties
in continuous and discrete-time settings for bilinear games to demonstrate
linear convergence to the Nash equilibrium. Finally, we empirically evaluate
our method on synthetic setups and CIFAR-10 image generation to demonstrate
improvements over baseline methods.
