Relaxing Bijectivity Constraints with Continuously Indexed Normalising
  Flows
  We show that normalising flows become pathological when used to model targets
whose supports have complicated topologies. In this scenario, we prove that a
flow must become arbitrarily numerically noninvertible in order to approximate
the target closely. This result has implications for all flow-based models, and
especially Residual Flows (ResFlows), which explicitly control the Lipschitz
constant of the bijection used. To address this, we propose Continuously
Indexed Flows (CIFs), which replace the single bijection used by normalising
flows with a continuously indexed family of bijections, and which can
intuitively "clean up" mass that would otherwise be misplaced by a single
bijection. We show theoretically that CIFs are not subject to the same
topological limitations as normalising flows, and obtain better empirical
performance on a variety of models and benchmarks.
