Generative Adversarial Network Training is a Continual Learning Problem
  Generative Adversarial Networks (GANs) have proven to be a powerful framework
for learning to draw samples from complex distributions. However, GANs are also
notoriously difficult to train, with mode collapse and oscillations a common
problem. We hypothesize that this is at least in part due to the evolution of
the generator distribution and the catastrophic forgetting tendency of neural
networks, which leads to the discriminator losing the ability to remember
synthesized samples from previous instantiations of the generator. Recognizing
this, our contributions are twofold. First, we show that GAN training makes for
a more interesting and realistic benchmark for continual learning methods
evaluation than some of the more canonical datasets. Second, we propose
leveraging continual learning techniques to augment the discriminator,
preserving its ability to recognize previous generator samples. We show that
the resulting methods add only a light amount of computation, involve minimal
changes to the model, and result in better overall performance on the examined
image and text generation tasks.
