Efficient Set-Valued Prediction in Multi-Class Classification
  In cases of uncertainty, a multi-class classifier preferably returns a set of
candidate classes instead of predicting a single class label with little
guarantee. More precisely, the classifier should strive for an optimal balance
between the correctness (the true class is among the candidates) and the
precision (the candidates are not too many) of its prediction. We formalize
this problem within a general decision-theoretic framework that unifies most of
the existing work in this area. In this framework, uncertainty is quantified in
terms of conditional class probabilities, and the quality of a predicted set is
measured in terms of a utility function. We then address the problem of finding
the Bayes-optimal prediction, i.e., the subset of class labels with highest
expected utility. For this problem, which is computationally challenging as
there are exponentially (in the number of classes) many predictions to choose
from, we propose efficient algorithms that can be applied to a broad family of
utility functions. Our theoretical results are complemented by experimental
studies, in which we analyze the proposed algorithms in terms of predictive
accuracy and runtime efficiency.
