Uncertainty Estimation in Autoregressive Structured Prediction
  Uncertainty estimation is important for ensuring safety and robustness of AI
systems. While most research in the area has focused on un-structured
prediction tasks, limited work has investigated general uncertainty estimation
approaches for structured prediction. Thus, this work aims to investigate
uncertainty estimation for autoregressive structured prediction tasks within a
single unified and interpretable probabilistic ensemble-based framework. We
consider: uncertainty estimation for sequence data at the token-level and
complete sequence-level; interpretations for, and applications of, various
measures of uncertainty; and discuss both the theoretical and practical
challenges associated with obtaining them. This work also provides baselines
for token-level and sequence-level error detection, and sequence-level
out-of-domain input detection on the WMT'14 English-French and WMT'17
English-German translation and LibriSpeech speech recognition datasets.
