Being Bayesian about Categorical Probability
  Neural networks utilize the softmax as a building block in classification
tasks, which contains an overconfidence problem and lacks an uncertainty
representation ability. As a Bayesian alternative to the softmax, we consider a
random variable of a categorical probability over class labels. In this
framework, the prior distribution explicitly models the presumed noise inherent
in the observed label, which provides consistent gains in generalization
performance in multiple challenging tasks. The proposed method inherits
advantages of Bayesian approaches that achieve better uncertainty estimation
and model calibration. Our method can be implemented as a plug-and-play loss
function with negligible computational overhead compared to the softmax with
the cross-entropy loss function.
