You Can't See the Forest for Its Trees: Assessing Deep Neural Network
  Testing via NeuraL Coverage
  This paper summarizes eight design requirements for DNN testing criteria,
taking into account distribution properties and practical concerns. We then
propose a new criterion, NLC, that satisfies all of these design requirements.
NLC treats a single DNN layer as the basic computational unit (rather than a
single neuron) and captures four critical features of neuron output
distributions. Thus, NLC is denoted as NeuraL Coverage, which more accurately
describes how neural networks comprehend inputs via approximated distributions
rather than neurons. We demonstrate that NLC is significantly correlated with
the diversity of a test suite across a number of tasks (classification and
generation) and data formats (image and text). Its capacity to discover DNN
prediction errors is promising. Test input mutation guided by NLC result in a
greater quality and diversity of exposed erroneous behaviors.
