Information Aware Max-Norm Dirichlet Networks for Predictive Uncertainty
  Estimation
  Precise estimation of uncertainty in predictions for AI systems is a critical
factor in ensuring trust and safety. Deep neural networks trained with a
conventional method are prone to over-confident predictions. In contrast to
Bayesian neural networks that learn approximate distributions on weights to
infer prediction confidence, we propose a novel method, Information Aware
Dirichlet networks, that learn an explicit Dirichlet prior distribution on
predictive distributions by minimizing a bound on the expected max norm of the
prediction error and penalizing information associated with incorrect outcomes.
Properties of the new cost function are derived to indicate how improved
uncertainty estimation is achieved. Experiments using real datasets show that
our technique outperforms, by a large margin, state-of-the-art neural networks
for estimating within-distribution and out-of-distribution uncertainty, and
detecting adversarial examples.
