Adaptive Weighted Discriminator for Training Generative Adversarial
  Networks
  Generative adversarial network (GAN) has become one of the most important
neural network models for classical unsupervised machine learning. A variety of
discriminator loss functions have been developed to train GAN's discriminators
and they all have a common structure: a sum of real and fake losses that only
depends on the actual and generated data respectively. One challenge associated
with an equally weighted sum of two losses is that the training may benefit one
loss but harm the other, which we show causes instability and mode collapse. In
this paper, we introduce a new family of discriminator loss functions that
adopts a weighted sum of real and fake parts, which we call adaptive weighted
loss functions or aw-loss functions. Using the gradients of the real and fake
parts of the loss, we can adaptively choose weights to train a discriminator in
the direction that benefits the GAN's stability. Our method can be potentially
applied to any discriminator model with a loss that is a sum of the real and
fake parts. Experiments validated the effectiveness of our loss functions on an
unconditional image generation task, improving the baseline results by a
significant margin on CIFAR-10, STL-10, and CIFAR-100 datasets in Inception
Scores and FID.
