TestRank: Bringing Order into Unlabeled Test Instances for Deep Learning
  Tasks
  Deep learning (DL) has achieved unprecedented success in a variety of tasks.
However, DL systems are notoriously difficult to test and debug due to the lack
of explainability of DL models and the huge test input space to cover.
Generally speaking, it is relatively easy to collect a massive amount of test
data, but the labeling cost can be quite high. Consequently, it is essential to
conduct test selection and label only those selected "high quality"
bug-revealing test inputs for test cost reduction.
  In this paper, we propose a novel test prioritization technique that brings
order into the unlabeled test instances according to their bug-revealing
capabilities, namely TestRank. Different from existing solutions, TestRank
leverages both intrinsic attributes and contextual attributes of test instances
when prioritizing them. To be specific, we first build a similarity graph on
test instances and training samples, and we conduct graph-based semi-supervised
learning to extract contextual features. Then, for a particular test instance,
the contextual features extracted from the graph neural network (GNN) and the
intrinsic features obtained with the DL model itself are combined to predict
its bug-revealing probability. Finally, TestRank prioritizes unlabeled test
instances in descending order of the above probability value. We evaluate the
performance of TestRank on a variety of image classification datasets.
Experimental results show that the debugging efficiency of our method
significantly outperforms existing test prioritization techniques.
