Sharp Bounds on the Approximation Rates, Metric Entropy, and $n$-widths
  of Shallow Neural Networks
  In this article, we study approximation properties of the variation spaces
corresponding to shallow neural networks with a variety of activation
functions. We introduce two main tools for estimating the metric entropy,
approximation rates, and $n$-widths of these spaces. First, we introduce the
notion of a smoothly parameterized dictionary and give upper bounds on the
non-linear approximation rates, metric entropy and $n$-widths of their absolute
convex hull. The upper bounds depend upon the order of smoothness of the
parameterization. This result is applied to dictionaries of ridge functions
corresponding to shallow neural networks, and they improve upon existing
results in many cases. Next, we provide a method for lower bounding the metric
entropy and $n$-widths of variation spaces which contain certain classes of
ridge functions. This result gives sharp lower bounds on the
$L^2$-approximation rates, metric entropy, and $n$-widths for variation spaces
corresponding to neural networks with a range of important activation
functions, including ReLU$^k$, sigmoidal activation functions with bounded
variation, and the B-spline activation functions.
