Efficient Conformal Prediction via Cascaded Inference with Expanded
  Admission
  In this paper, we present a novel approach for conformal prediction (CP), in
which we aim to identify a set of promising prediction candidates -- in place
of a single prediction. This set is guaranteed to contain a correct answer with
high probability, and is well-suited for many open-ended classification tasks.
In the standard CP paradigm, the predicted set can often be unusably large and
also costly to obtain. This is particularly pervasive in settings where the
correct answer is not unique, and the number of total possible answers is high.
We first expand the CP correctness criterion to allow for additional, inferred
"admissible" answers, which can substantially reduce the size of the predicted
set while still providing valid performance guarantees. Second, we amortize
costs by conformalizing prediction cascades, in which we aggressively prune
implausible labels early on by using progressively stronger classifiers --
again, while still providing valid performance guarantees. We demonstrate the
empirical effectiveness of our approach for multiple applications in natural
language processing and computational chemistry for drug discovery.
