Importance-Driven Deep Learning System Testing
  Deep Learning (DL) systems are key enablers for engineering intelligent
applications due to their ability to solve complex tasks such as image
recognition and machine translation. Nevertheless, using DL systems in safety-
and security-critical applications requires to provide testing evidence for
their dependable operation. Recent research in this direction focuses on
adapting testing criteria from traditional software engineering as a means of
increasing confidence for their correct behaviour. However, they are inadequate
in capturing the intrinsic properties exhibited by these systems. We bridge
this gap by introducing DeepImportance, a systematic testing methodology
accompanied by an Importance-Driven (IDC) test adequacy criterion for DL
systems. Applying IDC enables to establish a layer-wise functional
understanding of the importance of DL system components and use this
information to assess the semantic diversity of a test set. Our empirical
evaluation on several DL systems, across multiple DL datasets and with
state-of-the-art adversarial generation techniques demonstrates the usefulness
and effectiveness of DeepImportance and its ability to support the engineering
of more robust DL systems.
