Safety Verification of Deep Neural Networks
  Deep neural networks have achieved impressive experimental results in image
classification, but can surprisingly be unstable with respect to adversarial
perturbations, that is, minimal changes to the input image that cause the
network to misclassify it. With potential applications including perception
modules and end-to-end controllers for self-driving cars, this raises concerns
about their safety. We develop a novel automated verification framework for
feed-forward multi-layer neural networks based on Satisfiability Modulo Theory
(SMT). We focus on safety of image classification decisions with respect to
image manipulations, such as scratches or changes to camera angle or lighting
conditions that would result in the same class being assigned by a human, and
define safety for an individual decision in terms of invariance of the
classification within a small neighbourhood of the original image. We enable
exhaustive search of the region by employing discretisation, and propagate the
analysis layer by layer. Our method works directly with the network code and,
in contrast to existing methods, can guarantee that adversarial examples, if
they exist, are found for the given region and family of manipulations. If
found, adversarial examples can be shown to human testers and/or used to
fine-tune the network. We implement the techniques using Z3 and evaluate them
on state-of-the-art networks, including regularised and deep learning networks.
We also compare against existing techniques to search for adversarial examples
and estimate network robustness.
