The Hidden Uncertainty in a Neural Networks Activations
  The distribution of a neural network's latent representations has been
successfully used to detect out-of-distribution (OOD) data. This work
investigates whether this distribution moreover correlates with a model's
epistemic uncertainty, thus indicates its ability to generalise to novel
inputs. We first empirically verify that epistemic uncertainty can be
identified with the surprise, thus the negative log-likelihood, of observing a
particular latent representation. Moreover, we demonstrate that the
output-conditional distribution of hidden representations also allows
quantifying aleatoric uncertainty via the entropy of the predictive
distribution. We analyse epistemic and aleatoric uncertainty inferred from the
representations of different layers and conclude that deeper layers lead to
uncertainty with similar behaviour as established - but computationally more
expensive - methods (e.g. deep ensembles). While our approach does not require
modifying the training process, we follow prior work and experiment with an
additional regularising loss that increases the information in the latent
representations. We find that this leads to improved OOD detection of epistemic
uncertainty at the cost of ambiguous calibration close to the data
distribution. We verify our findings on both classification and regression
models.
