A statistical theory of cold posteriors in deep neural networks
  To get Bayesian neural networks to perform comparably to standard neural
networks it is usually necessary to artificially reduce uncertainty using a
"tempered" or "cold" posterior. This is extremely concerning: if the prior is
accurate, Bayes inference/decision theory is optimal, and any artificial
changes to the posterior should harm performance. While this suggests that the
prior may be at fault, here we argue that in fact, BNNs for image
classification use the wrong likelihood. In particular, standard image
benchmark datasets such as CIFAR-10 are carefully curated. We develop a
generative model describing curation which gives a principled Bayesian account
of cold posteriors, because the likelihood under this new generative model
closely matches the tempered likelihoods used in past work.
