Perturbative GAN: GAN with Perturbation Layers
  Perturbative GAN, which replaces convolution layers of existing convolutional
GANs (DCGAN, WGAN-GP, BIGGAN, etc.) with perturbation layers that adds a fixed
noise mask, is proposed. Compared with the convolu-tional GANs, the number of
parameters to be trained is smaller, the convergence of training is faster, the
incep-tion score of generated images is higher, and the overall training cost
is reduced. Algorithmic generation of the noise masks is also proposed, with
which the training, as well as the generation, can be boosted with hardware
acceleration. Perturbative GAN is evaluated using con-ventional datasets
(CIFAR10, LSUN, ImageNet), both in the cases when a perturbation layer is
adopted only for Generators and when it is introduced to both Generator and
Discriminator.
