A Noise-Sensitivity-Analysis-Based Test Prioritization Technique for
  Deep Neural Networks
  Deep neural networks (DNNs) have been widely used in the fields such as
natural language processing, computer vision and image recognition. But several
studies have been shown that deep neural networks can be easily fooled by
artificial examples with some perturbations, which are widely known as
adversarial examples. Adversarial examples can be used to attack deep neural
networks or to improve the robustness of deep neural networks. A common way of
generating adversarial examples is to first generate some noises and then add
them into original examples. In practice, different examples have different
noise-sensitive. To generate an effective adversarial example, it may be
necessary to add a lot of noise to low noise-sensitive example, which may make
the adversarial example meaningless. In this paper, we propose a
noise-sensitivity-analysis-based test prioritization technique to pick out
examples by their noise sensitivity. We construct an experiment to validate our
approach on four image sets and two DNN models, which shows that examples are
sensitive to noise and our method can effectively pick out examples by their
noise sensitivity.
