Universality of Deep Convolutional Neural Networks
  Deep learning has been widely applied and brought breakthroughs in speech
recognition, computer vision, and many other domains. The involved deep neural
network architectures and computational issues have been well studied in
machine learning. But there lacks a theoretical foundation for understanding
the approximation or generalization ability of deep learning methods generated
by the network architectures such as deep convolutional neural networks having
convolutional structures. Here we show that a deep convolutional neural network
(CNN) is universal, meaning that it can be used to approximate any continuous
function to an arbitrary accuracy when the depth of the neural network is large
enough. This answers an open question in learning theory. Our quantitative
estimate, given tightly in terms of the number of free parameters to be
computed, verifies the efficiency of deep CNNs in dealing with large
dimensional data. Our study also demonstrates the role of convolutions in deep
CNNs.
