Improving Confidence Estimation on Out-of-Domain Data for End-to-End
  Speech Recognition
  As end-to-end automatic speech recognition (ASR) models reach promising
performance, various downstream tasks rely on good confidence estimators for
these systems. Recent research has shown that model-based confidence estimators
have a significant advantage over using the output softmax probabilities. If
the input data to the speech recogniser is from mismatched acoustic and
linguistic conditions, the ASR performance and the corresponding confidence
estimators may exhibit severe degradation. Since confidence models are often
trained on the same in-domain data as the ASR, generalising to out-of-domain
(OOD) scenarios is challenging. By keeping the ASR model untouched, this paper
proposes two approaches to improve the model-based confidence estimators on OOD
data: using pseudo transcriptions and an additional OOD language model. With an
ASR model trained on LibriSpeech, experiments show that the proposed methods
can greatly improve the confidence metrics on TED-LIUM and Switchboard datasets
while preserving in-domain performance. Furthermore, the improved confidence
estimators are better calibrated on OOD data and can provide a much more
reliable criterion for data selection.
