Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling
  by Exploring Energy of the Discriminator
  Generative Adversarial Networks (GANs) have shown great promise in modeling
high dimensional data. The learning objective of GANs usually minimizes some
measure discrepancy, \textit{e.g.}, $f$-divergence~($f$-GANs) or Integral
Probability Metric~(Wasserstein GANs). With $f$-divergence as the objective
function, the discriminator essentially estimates the density ratio, and the
estimated ratio proves useful in further improving the sample quality of the
generator. However, how to leverage the information contained in the
discriminator of Wasserstein GANs (WGAN) is less explored. In this paper, we
introduce the Discriminator Contrastive Divergence, which is well motivated by
the property of WGAN's discriminator and the relationship between WGAN and
energy-based model. Compared to standard GANs, where the generator is directly
utilized to obtain new samples, our method proposes a semi-amortized generation
procedure where the samples are produced with the generator's output as an
initial state. Then several steps of Langevin dynamics are conducted using the
gradient of the discriminator. We demonstrate the benefits of significant
improved generation on both synthetic data and several real-world image
generation benchmarks.
