A Brief Introduction to Generative Models
  We introduce and motivate generative modeling as a central task for machine
learning and provide a critical view of the algorithms which have been proposed
for solving this task. We overview how generative modeling can be defined
mathematically as trying to make an estimating distribution the same as an
unknown ground truth distribution. This can then be quantified in terms of the
value of a statistical divergence between the two distributions. We outline the
maximum likelihood approach and how it can be interpreted as minimizing
KL-divergence. We explore a number of approaches in the maximum likelihood
family, while discussing their limitations. Finally, we explore the alternative
adversarial approach which involves studying the differences between an
estimating distribution and a real data distribution. We discuss how this
approach can give rise to new divergences and methods that are necessary to
make adversarial learning successful. We also discuss new evaluation metrics
which are required by the adversarial approach.
