On Convergence and Stability of GANs
  We propose studying GAN training dynamics as regret minimization, which is in
contrast to the popular view that there is consistent minimization of a
divergence between real and generated distributions. We analyze the convergence
of GAN training from this new point of view to understand why mode collapse
happens. We hypothesize the existence of undesirable local equilibria in this
non-convex game to be responsible for mode collapse. We observe that these
local equilibria often exhibit sharp gradients of the discriminator function
around some real data points. We demonstrate that these degenerate local
equilibria can be avoided with a gradient penalty scheme called DRAGAN. We show
that DRAGAN enables faster training, achieves improved stability with fewer
mode collapses, and leads to generator networks with better modeling
performance across a variety of architectures and objective functions.
