IDF++: Analyzing and Improving Integer Discrete Flows for Lossless
  Compression
  In this paper we analyse and improve integer discrete flows for lossless
compression. Integer discrete flows are a recently proposed class of models
that learn invertible transformations for integer-valued random variables.
Their discrete nature makes them particularly suitable for lossless compression
with entropy coding schemes. We start by investigating a recent theoretical
claim that states that invertible flows for discrete random variables are less
flexible than their continuous counterparts. We demonstrate with a proof that
this claim does not hold for integer discrete flows due to the embedding of
data with finite support into the countably infinite integer lattice.
Furthermore, we zoom in on the effect of gradient bias due to the
straight-through estimator in integer discrete flows, and demonstrate that its
influence is highly dependent on architecture choices and less prominent than
previously thought. Finally, we show how different architecture modifications
improve the performance of this model class for lossless compression, and that
they also enable more efficient compression: a model with half the number of
flow layers performs on par with or better than the original integer discrete
flow model.
