Implicit Rugosity Regularization via Data Augmentation
  Deep (neural) networks have been applied productively in a wide range of
supervised and unsupervised learning tasks. Unlike classical machine learning
algorithms, deep networks typically operate in the \emph{overparameterized}
regime, where the number of parameters is larger than the number of training
data points. Consequently, understanding the generalization properties and the
role of (explicit or implicit) regularization in these networks is of great
importance. In this work, we explore how the oft-used heuristic of \emph{data
augmentation} imposes an {\em implicit regularization} penalty of a novel
measure of the \emph{rugosity} or "roughness" based on the tangent Hessian of
the function fit to the training data.
