Closing the Dequantization Gap: PixelCNN as a Single-Layer Flow
  Flow models have recently made great progress at modeling ordinal discrete
data such as images and audio. Due to the continuous nature of flow models,
dequantization is typically applied when using them for such discrete data,
resulting in lower bound estimates of the likelihood. In this paper, we
introduce subset flows, a class of flows that can tractably transform finite
volumes and thus allow exact computation of likelihoods for discrete data.
Based on subset flows, we identify ordinal discrete autoregressive models,
including WaveNets, PixelCNNs and Transformers, as single-layer flows. We use
the flow formulation to compare models trained and evaluated with either the
exact likelihood or its dequantization lower bound. Finally, we study
multilayer flows composed of PixelCNNs and non-autoregressive coupling layers
and demonstrate state-of-the-art results on CIFAR-10 for flow models trained
with dequantization.
