Training Generative Adversarial Networks by Solving Ordinary
  Differential Equations
  The instability of Generative Adversarial Network (GAN) training has
frequently been attributed to gradient descent. Consequently, recent methods
have aimed to tailor the models and training procedures to stabilise the
discrete updates. In contrast, we study the continuous-time dynamics induced by
GAN training. Both theory and toy experiments suggest that these dynamics are
in fact surprisingly stable. From this perspective, we hypothesise that
instabilities in training GANs arise from the integration error in discretising
the continuous dynamics. We experimentally verify that well-known ODE solvers
(such as Runge-Kutta) can stabilise training - when combined with a regulariser
that controls the integration error. Our approach represents a radical
departure from previous methods which typically use adaptive optimisation and
stabilisation techniques that constrain the functional space (e.g. Spectral
Normalisation). Evaluation on CIFAR-10 and ImageNet shows that our method
outperforms several strong baselines, demonstrating its efficacy.
