Sobolev-type embeddings for neural network approximation spaces
  We consider neural network approximation spaces that classify functions
according to the rate at which they can be approximated (with error measured in
$L^p$) by ReLU neural networks with an increasing number of coefficients,
subject to bounds on the magnitude of the coefficients and the number of hidden
layers. We prove embedding theorems between these spaces for different values
of $p$. Furthermore, we derive sharp embeddings of these approximation spaces
into H\"older spaces. We find that, analogous to the case of classical function
spaces (such as Sobolev spaces, or Besov spaces) it is possible to trade
"smoothness" (i.e., approximation rate) for increased integrability.
  Combined with our earlier results in [arXiv:2104.02746], our embedding
theorems imply a somewhat surprising fact related to "learning" functions from
a given neural network space based on point samples: if accuracy is measured
with respect to the uniform norm, then an optimal "learning" algorithm for
reconstructing functions that are well approximable by ReLU neural networks is
simply given by piecewise constant interpolation on a tensor product grid.
