Approximating Probability Distributions by ReLU Networks
  How many neurons are needed to approximate a target probability distribution
using a neural network with a given input distribution and approximation error?
This paper examines this question for the case when the input distribution is
uniform, and the target distribution belongs to the class of histogram
distributions. We obtain a new upper bound on the number of required neurons,
which is strictly better than previously existing upper bounds. The key
ingredient in this improvement is an efficient construction of the neural nets
representing piecewise linear functions. We also obtain a lower bound on the
minimum number of neurons needed to approximate the histogram distributions.
