Confidence-Aware Learning for Deep Neural Networks
  Despite the power of deep neural networks for a wide range of tasks, an
overconfident prediction issue has limited their practical use in many
safety-critical applications. Many recent works have been proposed to mitigate
this issue, but most of them require either additional computational costs in
training and/or inference phases or customized architectures to output
confidence estimates separately. In this paper, we propose a method of training
deep neural networks with a novel loss function, named Correctness Ranking
Loss, which regularizes class probabilities explicitly to be better confidence
estimates in terms of ordinal ranking according to confidence. The proposed
method is easy to implement and can be applied to the existing architectures
without any modification. Also, it has almost the same computational costs for
training as conventional deep classifiers and outputs reliable predictions by a
single inference. Extensive experimental results on classification benchmark
datasets indicate that the proposed method helps networks to produce
well-ranked confidence estimates. We also demonstrate that it is effective for
the tasks closely related to confidence estimation, out-of-distribution
detection and active learning.
