Subsampling Generative Adversarial Networks: Density Ratio Estimation in
  Feature Space with Softplus Loss
  Filtering out unrealistic images from trained generative adversarial networks
(GANs) has attracted considerable attention recently. Two density ratio based
subsampling methods---Discriminator Rejection Sampling (DRS) and
Metropolis-Hastings GAN (MH-GAN)---were recently proposed, and their
effectiveness in improving GANs was demonstrated on multiple datasets. However,
DRS and MH-GAN are based on discriminator based density ratio estimation (DRE)
methods, so they may not work well if the discriminator in the trained GAN is
far from optimal. Moreover, they do not apply to some GANs (e.g., MMD-GAN). In
this paper, we propose a novel Softplus (SP) loss for DRE. Based on it, we
develop a sample-based DRE method in a feature space learned by a specially
designed and pre-trained ResNet-34 (DRE-F-SP). We derive the rate of
convergence of a density ratio model trained under the SP loss. Then, we
propose three different density ratio subsampling methods (DRE-F-SP+RS,
DRE-F-SP+MH, and DRE-F-SP+SIR) for GANs based on DRE-F-SP. Our subsampling
methods do not rely on the optimality of the discriminator and are suitable for
all types of GANs. We empirically show our subsampling approach can
substantially outperform DRS and MH-GAN on a synthetic dataset and the CIFAR-10
dataset, using multiple GANs.
