Uncertainty Estimation and Calibration with Finite-State Probabilistic
  RNNs
  Uncertainty quantification is crucial for building reliable and trustable
machine learning systems. We propose to estimate uncertainty in recurrent
neural networks (RNNs) via stochastic discrete state transitions over recurrent
timesteps. The uncertainty of the model can be quantified by running a
prediction several times, each time sampling from the recurrent state
transition distribution, leading to potentially different results if the model
is uncertain. Alongside uncertainty quantification, our proposed method offers
several advantages in different settings. The proposed method can (1) learn
deterministic and probabilistic automata from data, (2) learn well-calibrated
models on real-world classification tasks, (3) improve the performance of
out-of-distribution detection, and (4) control the exploration-exploitation
trade-off in reinforcement learning.
