Assessing Deep Neural Networks as Probability Estimators
  Deep Neural Networks (DNNs) have performed admirably in classification tasks.
However, the characterization of their classification uncertainties, required
for certain applications, has been lacking. In this work, we investigate the
issue by assessing DNNs' ability to estimate conditional probabilities and
propose a framework for systematic uncertainty characterization. Denoting the
input sample as x and the category as y, the classification task of assigning a
category y to a given input x can be reduced to the task of estimating the
conditional probabilities p(y|x), as approximated by the DNN at its last layer
using the softmax function. Since softmax yields a vector whose elements all
fall in the interval (0, 1) and sum to 1, it suggests a probabilistic
interpretation to the DNN's outcome. Using synthetic and real-world datasets,
we look into the impact of various factors, e.g., probability density f(x) and
inter-categorical sparsity, on the precision of DNNs' estimations of p(y|x),
and find that the likelihood probability density and the inter-categorical
sparsity have greater impacts than the prior probability to DNNs'
classification uncertainty.
