Training Generative Adversarial Networks with Adaptive Composite
  Gradient
  The wide applications of Generative adversarial networks benefit from the
successful training methods, guaranteeing that an object function converges to
the local minima. Nevertheless, designing an efficient and competitive training
method is still a challenging task due to the cyclic behaviors of some
gradient-based ways and the expensive computational cost of these methods based
on the Hessian matrix. This paper proposed the adaptive Composite Gradients
(ACG) method, linearly convergent in bilinear games under suitable settings.
Theory and toy-function experiments suggest that our approach can alleviate the
cyclic behaviors and converge faster than recently proposed algorithms.
Significantly, the ACG method is not only used to find stable fixed points in
bilinear games as well as in general games. The ACG method is a novel
semi-gradient-free algorithm since it does not need to calculate the gradient
of each step, reducing the computational cost of gradient and Hessian by
utilizing the predictive information in future iterations. We conducted two
mixture of Gaussians experiments by integrating ACG to existing algorithms with
Linear GANs. Results show ACG is competitive with the previous algorithms.
Realistic experiments on four prevalent data sets (MNIST, Fashion-MNIST,
CIFAR-10, and CelebA) with DCGANs show that our ACG method outperforms several
baselines, which illustrates the superiority and efficacy of our method.
