Noise-Response Analysis of Deep Neural Networks Quantifies Robustness
  and Fingerprints Structural Malware
  The ubiquity of deep neural networks (DNNs), cloud-based training, and
transfer learning is giving rise to a new cybersecurity frontier in which
unsecure DNNs have `structural malware' (i.e., compromised weights and
activation pathways). In particular, DNNs can be designed to have backdoors
that allow an adversary to easily and reliably fool an image classifier by
adding a pattern of pixels called a trigger. It is generally difficult to
detect backdoors, and existing detection methods are computationally expensive
and require extensive resources (e.g., access to the training data). Here, we
propose a rapid feature-generation technique that quantifies the robustness of
a DNN, `fingerprints' its nonlinearity, and allows us to detect backdoors (if
present). Our approach involves studying how a DNN responds to noise-infused
images with varying noise intensity, which we summarize with titration curves.
We find that DNNs with backdoors are more sensitive to input noise and respond
in a characteristic way that reveals the backdoor and where it leads (its
`target'). Our empirical results demonstrate that we can accurately detect
backdoors with high confidence orders-of-magnitude faster than existing
approaches (seconds versus hours).
