Size-Noise Tradeoffs in Generative Networks
  This paper investigates the ability of generative networks to convert their
input noise distributions into other distributions. Firstly, we demonstrate a
construction that allows ReLU networks to increase the dimensionality of their
noise distribution by implementing a "space-filling" function based on iterated
tent maps. We show this construction is optimal by analyzing the number of
affine pieces in functions computed by multivariate ReLU networks. Secondly, we
provide efficient ways (using polylog $(1/\epsilon)$ nodes) for networks to
pass between univariate uniform and normal distributions, using a Taylor series
approximation and a binary search gadget for computing function inverses.
Lastly, we indicate how high dimensional distributions can be efficiently
transformed into low dimensional distributions.
