Detecting Backdoors in Neural Networks Using Novel Feature-Based Anomaly
  Detection
  This paper proposes a new defense against neural network backdooring attacks
that are maliciously trained to mispredict in the presence of attacker-chosen
triggers. Our defense is based on the intuition that the feature extraction
layers of a backdoored network embed new features to detect the presence of a
trigger and the subsequent classification layers learn to mispredict when
triggers are detected. Therefore, to detect backdoors, the proposed defense
uses two synergistic anomaly detectors trained on clean validation data: the
first is a novelty detector that checks for anomalous features, while the
second detects anomalous mappings from features to outputs by comparing with a
separate classifier trained on validation data. The approach is evaluated on a
wide range of backdoored networks (with multiple variations of triggers) that
successfully evade state-of-the-art defenses. Additionally, we evaluate the
robustness of our approach on imperceptible perturbations, scalability on
large-scale datasets, and effectiveness under domain shift. This paper also
shows that the defense can be further improved using data augmentation.
