Energy-Based Open-World Uncertainty Modeling for Confidence Calibration
  Confidence calibration is of great importance to the reliability of decisions
made by machine learning systems. However, discriminative classifiers based on
deep neural networks are often criticized for producing overconfident
predictions that fail to reflect the true correctness likelihood of
classification accuracy. We argue that such an inability to model uncertainty
is mainly caused by the closed-world nature in softmax: a model trained by the
cross-entropy loss will be forced to classify input into one of $K$ pre-defined
categories with high probability. To address this problem, we for the first
time propose a novel $K$+1-way softmax formulation, which incorporates the
modeling of open-world uncertainty as the extra dimension. To unify the
learning of the original $K$-way classification task and the extra dimension
that models uncertainty, we propose a novel energy-based objective function,
and moreover, theoretically prove that optimizing such an objective essentially
forces the extra dimension to capture the marginal data distribution. Extensive
experiments show that our approach, Energy-based Open-World Softmax
(EOW-Softmax), is superior to existing state-of-the-art methods in improving
confidence calibration.
