Loss-Calibrated Approximate Inference in Bayesian Neural Networks
  Current approaches in approximate inference for Bayesian neural networks
minimise the Kullback-Leibler divergence to approximate the true posterior over
the weights. However, this approximation is without knowledge of the final
application, and therefore cannot guarantee optimal predictions for a given
task. To make more suitable task-specific approximations, we introduce a new
loss-calibrated evidence lower bound for Bayesian neural networks in the
context of supervised learning, informed by Bayesian decision theory. By
introducing a lower bound that depends on a utility function, we ensure that
our approximation achieves higher utility than traditional methods for
applications that have asymmetric utility functions. Furthermore, in using
dropout inference, we highlight that our new objective is identical to that of
standard dropout neural networks, with an additional utility-dependent penalty
term. We demonstrate our new loss-calibrated model with an illustrative medical
example and a restricted model capacity experiment, and highlight failure modes
of the comparable weighted cross entropy approach. Lastly, we demonstrate the
scalability of our method to real world applications with per-pixel semantic
segmentation on an autonomous driving data set.
