Backdoor Scanning for Deep Neural Networks through K-Arm Optimization
  Back-door attack poses a severe threat to deep learning systems. It injects
hidden malicious behaviors to a model such that any input stamped with a
special pattern can trigger such behaviors. Detecting back-door is hence of
pressing need. Many existing defense techniques use optimization to generate
the smallest input pattern that forces the model to misclassify a set of benign
inputs injected with the pattern to a target label. However, the complexity is
quadratic to the number of class labels such that they can hardly handle models
with many classes. Inspired by Multi-Arm Bandit in Reinforcement Learning, we
propose a K-Arm optimization method for backdoor detection. By iteratively and
stochastically selecting the most promising labels for optimization with the
guidance of an objective function, we substantially reduce the complexity,
allowing to handle models with many classes. Moreover, by iteratively refining
the selection of labels to optimize, it substantially mitigates the uncertainty
in choosing the right labels, improving detection accuracy. At the time of
submission, the evaluation of our method on over 4000 models in the IARPA
TrojAI competition from round 1 to the latest round 4 achieves top performance
on the leaderboard. Our technique also supersedes three state-of-the-art
techniques in terms of accuracy and the scanning time needed.
