On the Turnpike to Design of Deep Neural Nets: Explicit Depth Bounds
  It is well-known that the training of Deep Neural Networks (DNN) can be
formalized in the language of optimal control. In this context, this paper
leverages classical turnpike properties of optimal control problems to attempt
a quantifiable answer to the question of how many layers should be considered
in a DNN. The underlying assumption is that the number of neurons per layer --
i.e., the width of the DNN -- is kept constant. Pursuing a different route than
the classical analysis of approximation properties of sigmoidal functions, we
prove explicit bounds on the required depths of DNNs based on asymptotic
reachability assumptions and a dissipativity-inducing choice of the
regularization terms in the training problem. Numerical results obtained for
the two spiral task data set for classification indicate that the proposed
estimates can provide non-conservative depth bounds.
