Approximation Properties of Deep ReLU CNNs
  This paper is devoted to establishing $L^2$ approximation properties for deep
ReLU convolutional neural networks (CNNs) on two-dimensional space. The
analysis is based on a decomposition theorem for convolutional kernels with
large spatial size and multi-channel. Given that decomposition and the property
of the ReLU activation function, a universal approximation theorem of deep ReLU
CNNs with classic structure is obtained by showing its connection with ReLU
deep neural networks (DNNs) with one hidden layer. Furthermore, approximation
properties are also obtained for neural networks with ResNet, pre-act ResNet,
and MgNet architecture based on connections between these networks.
