How (not) to Train your Generative Model: Scheduled Sampling,
  Likelihood, Adversary?
  Modern applications and progress in deep learning research have created
renewed interest for generative models of text and of images. However, even
today it is unclear what objective functions one should use to train and
evaluate these models. In this paper we present two contributions.
  Firstly, we present a critique of scheduled sampling, a state-of-the-art
training method that contributed to the winning entry to the MSCOCO image
captioning benchmark in 2015. Here we show that despite this impressive
empirical performance, the objective function underlying scheduled sampling is
improper and leads to an inconsistent learning algorithm.
  Secondly, we revisit the problems that scheduled sampling was meant to
address, and present an alternative interpretation. We argue that maximum
likelihood is an inappropriate training objective when the end-goal is to
generate natural-looking samples. We go on to derive an ideal objective
function to use in this situation instead. We introduce a generalisation of
adversarial training, and show how such method can interpolate between maximum
likelihood training and our ideal training objective. To our knowledge this is
the first theoretical analysis that explains why adversarial training tends to
produce samples with higher perceived quality.
