Label-Descriptive Patterns and Their Application to Characterizing
  Classification Errors
  State-of-the-art deep learning methods achieve human-like performance on many
tasks, but make errors nevertheless. Characterizing these errors in easily
interpretable terms gives insight into whether a classifier is prone to making
systematic errors, but also gives a way to act and improve the classifier. We
propose to discover those feature-value combinations (ie. patterns) that
strongly correlate with correct resp. erroneous predictions to obtain a global
and interpretable description for arbitrary classifiers. We show this is an
instance of the more general label description problem, which we formulate in
terms of the Minimum Description Length principle. To discover a good pattern
set, we develop the efficient Premise algorithm. Through an extensive set of
experiments we show it performs very well in practice on both synthetic and
real-world data. Unlike existing solutions, it ably recovers ground truth
patterns, even on highly imbalanced data over many features. Through two case
studies on Visual Question Answering and Named Entity Recognition, we confirm
that Premise gives clear and actionable insight into the systematic errors made
by modern NLP classifiers.
