The Inductive Bias of Restricted f-GANs
  Generative adversarial networks are a novel method for statistical inference
that have achieved much empirical success; however, the factors contributing to
this success remain ill-understood. In this work, we attempt to analyze
generative adversarial learning -- that is, statistical inference as the result
of a game between a generator and a discriminator -- with the view of
understanding how it differs from classical statistical inference solutions
such as maximum likelihood inference and the method of moments.
  Specifically, we provide a theoretical characterization of the distribution
inferred by a simple form of generative adversarial learning called restricted
f-GANs -- where the discriminator is a function in a given function class, the
distribution induced by the generator is restricted to lie in a pre-specified
distribution class and the objective is similar to a variational form of the
f-divergence. A consequence of our result is that for linear KL-GANs -- that
is, when the discriminator is a linear function over some feature space and f
corresponds to the KL-divergence -- the distribution induced by the optimal
generator is neither the maximum likelihood nor the method of moments solution,
but an interesting combination of both.
