Solve Minimax Optimization by Anderson Acceleration
  Many modern machine learning algorithms such as generative adversarial
networks (GANs) and adversarial training can be formulated as minimax
optimization. Gradient descent ascent (GDA) is the most commonly used algorithm
due to its simplicity. However, GDA can converge to non-optimal minimax points.
We propose a new minimax optimization framework, GDA-AM, that views the
GDAdynamics as a fixed-point iteration and solves it using Anderson Mixing to
con-verge to the local minimax. It addresses the diverging issue of
simultaneous GDAand accelerates the convergence of alternating GDA. We show
theoretically that the algorithm can achieve global convergence for bilinear
problems under mild conditions. We also empirically show that GDA-AMsolves a
variety of minimax problems and improves GAN training on several datasets
