Laplacian Networks: Bounding Indicator Function Smoothness for Neural
  Network Robustness
  For the past few years, Deep Neural Network (DNN) robustness has become a
question of paramount importance. As a matter of fact, in sensitive settings
misclassification can lead to dramatic consequences. Such misclassifications
are likely to occur when facing adversarial attacks, hardware failures or
limitations, and imperfect signal acquisition. To address this question,
authors have proposed different approaches aiming at increasing the robustness
of DNNs, such as adding regularizers or training using noisy examples. In this
paper we propose a new regularizer built upon the Laplacian of similarity
graphs obtained from the representation of training data at each layer of the
DNN architecture. This regularizer penalizes large changes (across consecutive
layers in the architecture) in the distance between examples of different
classes, and as such enforces smooth variations of the class boundaries. Since
it is agnostic to the type of deformations that are expected when predicting
with the DNN, the proposed regularizer can be combined with existing ad-hoc
methods. We provide theoretical justification for this regularizer and
demonstrate its effectiveness to improve robustness of DNNs on classical
supervised learning vision datasets.
