Radius-margin bounds for deep neural networks
  Explaining the unreasonable effectiveness of deep learning has eluded
researchers around the globe. Various authors have described multiple metrics
to evaluate the capacity of deep architectures. In this paper, we allude to the
radius margin bounds described for a support vector machine (SVM) with hinge
loss, apply the same to the deep feed-forward architectures and derive the
Vapnik-Chervonenkis (VC) bounds which are different from the earlier bounds
proposed in terms of number of weights of the network. In doing so, we also
relate the effectiveness of techniques like Dropout and Dropconnect in bringing
down the capacity of the network. Finally, we describe the effect of maximizing
the input as well as the output margin to achieve an input noise-robust deep
architecture.
