Quantifying Uncertainty in Deep Learning Approaches to Radio Galaxy
  Classification
  In this work we use variational inference to quantify the degree of
uncertainty in deep learning model predictions of radio galaxy classification.
We show that the level of model posterior variance for individual test samples
is correlated with human uncertainty when labelling radio galaxies. We explore
the model performance and uncertainty calibration for different weight priors
and suggest that a sparse prior produces more well-calibrated uncertainty
estimates. Using the posterior distributions for individual weights, we
demonstrate that we can prune 30% of the fully-connected layer weights without
significant loss of performance by removing the weights with the lowest
signal-to-noise ratio. A larger degree of pruning can be achieved using a
Fisher information based ranking, but both pruning methods affect the
uncertainty calibration for Fanaroff-Riley type I and type II radio galaxies
differently. Like other work in this field, we experience a cold posterior
effect, whereby the posterior must be down-weighted to achieve good predictive
performance. We examine whether adapting the cost function to accommodate model
misspecification can compensate for this effect, but find that it does not make
a significant difference. We also examine the effect of principled data
augmentation and find that this improves upon the baseline but also does not
compensate for the observed effect. We interpret this as the cold posterior
effect being due to the overly effective curation of our training sample
leading to likelihood misspecification, and raise this as a potential issue for
Bayesian deep learning approaches to radio galaxy classification in future.
