Construction of neural networks for realization of localized deep
  learning
  The subject of deep learning has recently attracted users of machine learning
from various disciplines, including: medical diagnosis and bioinformatics,
financial market analysis and online advertisement, speech and handwriting
recognition, computer vision and natural language processing, time series
forecasting, and search engines. However, theoretical development of deep
learning is still at its infancy. The objective of this paper is to introduce a
deep neural network (also called deep-net) approach to localized manifold
learning, with each hidden layer endowed with a specific learning task. For the
purpose of illustrations, we only focus on deep-nets with three hidden layers,
with the first layer for dimensionality reduction, the second layer for bias
reduction, and the third layer for variance reduction. A feedback component
also designed to eliminate outliers. The main theoretical result in this paper
is the order $\mathcal O\left(m^{-2s/(2s+d)}\right)$ of approximation of the
regression function with regularity $s$, in terms of the number $m$ of sample
points, where the (unknown) manifold dimension $d$ replaces the dimension $D$
of the sampling (Euclidean) space for shallow nets.
