A function space analysis of finite neural networks with insights from
  sampling theory
  This work suggests using sampling theory to analyze the function space
represented by neural networks. First, it shows, under the assumption of a
finite input domain, which is the common case in training neural networks, that
the function space generated by multi-layer networks with non-expansive
activation functions is smooth. This extends over previous works that show
results for the case of infinite width ReLU networks. Then, under the
assumption that the input is band-limited, we provide novel error bounds for
univariate neural networks. We analyze both deterministic uniform and random
sampling showing the advantage of the former.
