Full error analysis for the training of deep neural networks
  Deep learning algorithms have been applied very successfully in recent years
to a range of problems out of reach for classical solution paradigms.
Nevertheless, there is no completely rigorous mathematical error and
convergence analysis which explains the success of deep learning algorithms.
The error of a deep learning algorithm can in many situations be decomposed
into three parts, the approximation error, the generalization error, and the
optimization error. In this work we estimate for a certain deep learning
algorithm each of these three errors and combine these three error estimates to
obtain an overall error analysis for the deep learning algorithm under
consideration. In particular, we thereby establish convergence with a suitable
convergence speed for the overall error of the deep learning algorithm under
consideration. Our convergence speed analysis is far from optimal and the
convergence speed that we establish is rather slow, increases exponentially in
the dimensions, and, in particular, suffers from the curse of dimensionality.
The main contribution of this work is, instead, to provide a full error
analysis (i) which covers each of the three different sources of errors usually
emerging in deep learning algorithms and (ii) which merges these three sources
of errors into one overall error estimate for the considered deep learning
algorithm.
