Robustness of Neural Networks: A Probabilistic and Practical Approach
  Neural networks are becoming increasingly prevalent in software, and it is
therefore important to be able to verify their behavior. Because verifying the
correctness of neural networks is extremely challenging, it is common to focus
on the verification of other properties of these systems. One important
property, in particular, is robustness. Most existing definitions of
robustness, however, focus on the worst-case scenario where the inputs are
adversarial. Such notions of robustness are too strong, and unlikely to be
satisfied by-and verifiable for-practical neural networks. Observing that
real-world inputs to neural networks are drawn from non-adversarial probability
distributions, we propose a novel notion of robustness: probabilistic
robustness, which requires the neural network to be robust with at least $(1 -
\epsilon)$ probability with respect to the input distribution. This
probabilistic approach is practical and provides a principled way of estimating
the robustness of a neural network. We also present an algorithm, based on
abstract interpretation and importance sampling, for checking whether a neural
network is probabilistically robust. Our algorithm uses abstract interpretation
to approximate the behavior of a neural network and compute an
overapproximation of the input regions that violate robustness. It then uses
importance sampling to counter the effect of such overapproximation and compute
an accurate estimate of the probability that the neural network violates the
robustness property.
