Robustness Certificates Against Adversarial Examples for ReLU Networks
  While neural networks have achieved high performance in different learning
tasks, their accuracy drops significantly in the presence of small adversarial
perturbations to inputs. Defenses based on regularization and adversarial
training are often followed by new attacks to defeat them. In this paper, we
propose attack-agnostic robustness certificates for a multi-label
classification problem using a deep ReLU network. Although computing the exact
distance of a given input sample to the classification decision boundary
requires solving a non-convex optimization, we characterize two lower bounds
for such distances, namely the simplex certificate and the decision boundary
certificate. These robustness certificates leverage the piece-wise linear
structure of ReLU networks and use the fact that in a polyhedron around a given
sample, the prediction function is linear. In particular, the proposed simplex
certificate has a closed-form, is differentiable and is an order of magnitude
faster to compute than the existing methods even for deep networks. In addition
to theoretical bounds, we provide numerical results for our certificates over
MNIST and compare them with some existing upper bounds.
