Small-GAN: Speeding Up GAN Training Using Core-sets
  Recent work by Brock et al. (2018) suggests that Generative Adversarial
Networks (GANs) benefit disproportionately from large mini-batch sizes.
Unfortunately, using large batches is slow and expensive on conventional
hardware. Thus, it would be nice if we could generate batches that were
effectively large though actually small. In this work, we propose a method to
do this, inspired by the use of Coreset-selection in active learning. When
training a GAN, we draw a large batch of samples from the prior and then
compress that batch using Coreset-selection. To create effectively large
batches of 'real' images, we create a cached dataset of Inception activations
of each training image, randomly project them down to a smaller dimension, and
then use Coreset-selection on those projected activations at training time. We
conduct experiments showing that this technique substantially reduces training
time and memory usage for modern GAN variants, that it reduces the fraction of
dropped modes in a synthetic dataset, and that it allows GANs to reach a new
state of the art in anomaly detection.
