Adversarial symmetric GANs: bridging adversarial samples and adversarial
  networks
  Generative adversarial networks have achieved remarkable performance on
various tasks but suffer from training instability. Despite many training
strategies proposed to improve training stability, this issue remains as a
challenge. In this paper, we investigate the training instability from the
perspective of adversarial samples and reveal that adversarial training on fake
samples is implemented in vanilla GANs, but adversarial training on real
samples has long been overlooked. Consequently, the discriminator is extremely
vulnerable to adversarial perturbation and the gradient given by the
discriminator contains non-informative adversarial noises, which hinders the
generator from catching the pattern of real samples. Here, we develop
adversarial symmetric GANs (AS-GANs) that incorporate adversarial training of
the discriminator on real samples into vanilla GANs, making adversarial
training symmetrical. The discriminator is therefore more robust and provides
more informative gradient with less adversarial noise, thereby stabilizing
training and accelerating convergence. The effectiveness of the AS-GANs is
verified on image generation on CIFAR-10 , CelebA, and LSUN with varied network
architectures. Not only the training is more stabilized, but the FID scores of
generated samples are consistently improved by a large margin compared to the
baseline. The bridging of adversarial samples and adversarial networks provides
a new approach to further develop adversarial networks.
