Negative Momentum for Improved Game Dynamics
  Games generalize the single-objective optimization paradigm by introducing
different objective functions for different players. Differentiable games often
proceed by simultaneous or alternating gradient updates. In machine learning,
games are gaining new importance through formulations like generative
adversarial networks (GANs) and actor-critic systems. However, compared to
single-objective optimization, game dynamics are more complex and less
understood. In this paper, we analyze gradient-based methods with momentum on
simple games. We prove that alternating updates are more stable than
simultaneous updates. Next, we show both theoretically and empirically that
alternating gradient updates with a negative momentum term achieves convergence
in a difficult toy adversarial problem, but also on the notoriously difficult
to train saturating GANs.
