NanoFlow: Scalable Normalizing Flows with Sublinear Parameter Complexity
  Normalizing flows (NFs) have become a prominent method for deep generative
models that allow for an analytic probability density estimation and efficient
synthesis. However, a flow-based network is considered to be inefficient in
parameter complexity because of reduced expressiveness of bijective mapping,
which renders the models unfeasibly expensive in terms of parameters. We
present an alternative parameterization scheme called NanoFlow, which uses a
single neural density estimator to model multiple transformation stages. Hence,
we propose an efficient parameter decomposition method and the concept of flow
indication embedding, which are key missing components that enable density
estimation from a single neural network. Experiments performed on audio and
image models confirm that our method provides a new parameter-efficient
solution for scalable NFs with significant sublinear parameter complexity.
