Towards a Better Global Loss Landscape of GANs
  Understanding of GAN training is still very limited. One major challenge is
its non-convex-non-concave min-max objective, which may lead to sub-optimal
local minima. In this work, we perform a global landscape analysis of the
empirical loss of GANs. We prove that a class of separable-GAN, including the
original JS-GAN, has exponentially many bad basins which are perceived as
mode-collapse. We also study the relativistic pairing GAN (RpGAN) loss which
couples the generated samples and the true samples. We prove that RpGAN has no
bad basins. Experiments on synthetic data show that the predicted bad basin can
indeed appear in training. We also perform experiments to support our theory
that RpGAN has a better landscape than separable-GAN. For instance, we
empirically show that RpGAN performs better than separable-GAN with relatively
narrow neural nets. The code is available at https://github.com/AilsaF/RS-GAN.
