Entropic GANs meet VAEs: A Statistical Approach to Compute Sample
  Likelihoods in GANs
  Building on the success of deep learning, two modern approaches to learn a
probability model from the data are Generative Adversarial Networks (GANs) and
Variational AutoEncoders (VAEs). VAEs consider an explicit probability model
for the data and compute a generative distribution by maximizing a variational
lower-bound on the log-likelihood function. GANs, however, compute a generative
model by minimizing a distance between observed and generated probability
distributions without considering an explicit model for the observed data. The
lack of having explicit probability models in GANs prohibits computation of
sample likelihoods in their frameworks and limits their use in statistical
inference problems. In this work, we resolve this issue by constructing an
explicit probability model that can be used to compute sample likelihood
statistics in GANs. In particular, we prove that under this probability model,
a family of Wasserstein GANs with an entropy regularization can be viewed as a
generative model that maximizes a variational lower-bound on average sample log
likelihoods, an approach that VAEs are based on. This result makes a principled
connection between two modern generative models, namely GANs and VAEs. In
addition to the aforementioned theoretical results, we compute likelihood
statistics for GANs trained on Gaussian, MNIST, SVHN, CIFAR-10 and LSUN
datasets. Our numerical results validate the proposed theory.
