DeepSmartFuzzer: Reward Guided Test Generation For Deep Learning
  Testing Deep Neural Network (DNN) models has become more important than ever
with the increasing usage of DNN models in safety-critical domains such as
autonomous cars. The traditional approach of testing DNNs is to create a test
set, which is a random subset of the dataset about the problem of interest.
This kind of approach is not enough for testing most of the real-world
scenarios since these traditional test sets do not include corner cases, while
a corner case input is generally considered to introduce erroneous behaviors.
Recent works on adversarial input generation, data augmentation, and
coverage-guided fuzzing (CGF) have provided new ways to extend traditional test
sets. Among those, CGF aims to produce new test inputs by fuzzing existing ones
to achieve high coverage on a test adequacy criterion (i.e. coverage
criterion). Given that the subject test adequacy criterion is a
well-established one, CGF can potentially find error inducing inputs for
different underlying reasons. In this paper, we propose a novel CGF solution
for structural testing of DNNs. The proposed fuzzer employs Monte Carlo Tree
Search to drive the coverage-guided search in the pursuit of achieving high
coverage. Our evaluation shows that the inputs generated by our method result
in higher coverage than the inputs produced by the previously introduced
coverage-guided fuzzing techniques.
