Bounds on the Approximation Power of Feedforward Neural Networks
  The approximation power of general feedforward neural networks with piecewise
linear activation functions is investigated. First, lower bounds on the size of
a network are established in terms of the approximation error and network depth
and width. These bounds improve upon state-of-the-art bounds for certain
classes of functions, such as strongly convex functions. Second, an upper bound
is established on the difference of two neural networks with identical weights
but different activation functions.
