Constructive Universal High-Dimensional Distribution Generation through
  Deep ReLU Networks
  We present an explicit deep neural network construction that transforms
uniformly distributed one-dimensional noise into an arbitrarily close
approximation of any two-dimensional Lipschitz-continuous target distribution.
The key ingredient of our design is a generalization of the "space-filling"
property of sawtooth functions discovered in (Bailey & Telgarsky, 2018). We
elicit the importance of depth - in our neural network construction - in
driving the Wasserstein distance between the target distribution and the
approximation realized by the network to zero. An extension to output
distributions of arbitrary dimension is outlined. Finally, we show that the
proposed construction does not incur a cost - in terms of error measured in
Wasserstein-distance - relative to generating $d$-dimensional target
distributions from $d$ independent random variables.
