Approximation properties of Residual Neural Networks for Kolmogorov PDEs
  In recent years residual neural networks (ResNets) as introduced by [He, K.,
Zhang, X., Ren, S., and Sun, J., Proceedings of the IEEE conference on computer
vision and pattern recognition (2016), 770-778] have become very popular in a
large number of applications, including in image classification and
segmentation. They provide a new perspective in training very deep neural
networks without suffering the vanishing gradient problem. In this article we
show that ResNets are able to approximate solutions of Kolmogorov partial
differential equations (PDEs) with constant diffusion and possibly nonlinear
drift coefficients without suffering the curse of dimensionality, which is to
say the number of parameters of the approximating ResNets grows at most
polynomially in the reciprocal of the approximation accuracy $\varepsilon > 0$
and the dimension of the considered PDE $d\in\mathbb{N}$. We adapt a proof in
[Jentzen, A., Salimova, D., and Welti, T., Commun. Math. Sci. 19, 5 (2021),
1167-1205] - who showed a similar result for feedforward neural networks (FNNs)
- to ResNets. In contrast to FNNs, the Euler-Maruyama approximation structure
of ResNets simplifies the construction of the approximating ResNets
substantially. Moreover, contrary to the above work, in our proof using ResNets
does not require the existence of an FNN (or a ResNet) representing the
identity map, which enlarges the set of applicable activation functions.
