Training GANs with Optimism
  We address the issue of limit cycling behavior in training Generative
Adversarial Networks and propose the use of Optimistic Mirror Decent (OMD) for
training Wasserstein GANs. Recent theoretical results have shown that
optimistic mirror decent (OMD) can enjoy faster regret rates in the context of
zero-sum games. WGANs is exactly a context of solving a zero-sum game with
simultaneous no-regret dynamics. Moreover, we show that optimistic mirror
decent addresses the limit cycling problem in training WGANs. We formally show
that in the case of bi-linear zero-sum games the last iterate of OMD dynamics
converges to an equilibrium, in contrast to GD dynamics which are bound to
cycle. We also portray the huge qualitative difference between GD and OMD
dynamics with toy examples, even when GD is modified with many adaptations
proposed in the recent literature, such as gradient penalty or momentum. We
apply OMD WGAN training to a bioinformatics problem of generating DNA
sequences. We observe that models trained with OMD achieve consistently smaller
KL divergence with respect to the true underlying distribution, than models
trained with GD variants. Finally, we introduce a new algorithm, Optimistic
Adam, which is an optimistic variant of Adam. We apply it to WGAN training on
CIFAR10 and observe improved performance in terms of inception score as
compared to Adam.
