A Priori Estimates of the Population Risk for Residual Networks
  Optimal a priori estimates are derived for the population risk, also known as
the generalization error, of a regularized residual network model. An important
part of the regularized model is the usage of a new path norm, called the
weighted path norm, as the regularization term. The weighted path norm treats
the skip connections and the nonlinearities differently so that paths with more
nonlinearities are regularized by larger weights. The error estimates are a
priori in the sense that the estimates depend only on the target function, not
on the parameters obtained in the training process. The estimates are optimal,
in a high dimensional setting, in the sense that both the bound for the
approximation and estimation errors are comparable to the Monte Carlo error
rates. A crucial step in the proof is to establish an optimal bound for the
Rademacher complexity of the residual networks. Comparisons are made with
existing norm-based generalization error bounds.
