Autoregressive Models: What Are They Good For?
  Autoregressive (AR) models have become a popular tool for unsupervised
learning, achieving state-of-the-art log likelihood estimates. We investigate
the use of AR models as density estimators in two settings -- as a learning
signal for image translation, and as an outlier detector -- and find that these
density estimates are much less reliable than previously thought. We examine
the underlying optimization issues from both an empirical and theoretical
perspective, and provide a toy example that illustrates the problem.
Overwhelmingly, we find that density estimates do not correlate with perceptual
quality and are unhelpful for downstream tasks.
