Radio Galaxy Zoo: Using semi-supervised learning to leverage large
  unlabelled data-sets for radio galaxy classification under data-set shift
  In this work we examine the classification accuracy and robustness of a
state-of-the-art semi-supervised learning (SSL) algorithm applied to the
morphological classification of radio galaxies. We test if SSL with fewer
labels can achieve test accuracies comparable to the supervised
state-of-the-art and whether this holds when incorporating previously unseen
data. We find that for the radio galaxy classification problem considered, SSL
provides additional regularisation and outperforms the baseline test accuracy.
However, in contrast to model performance metrics reported on computer science
benchmarking data-sets, we find that improvement is limited to a narrow range
of label volumes, with performance falling off rapidly at low label volumes.
Additionally, we show that SSL does not improve model calibration, regardless
of whether classification is improved. Moreover, we find that when different
underlying catalogues drawn from the same radio survey are used to provide the
labelled and unlabelled data-sets required for SSL, a significant drop in
classification performance is observered, highlighting the difficulty of
applying SSL techniques under dataset shift. We show that a class-imbalanced
unlabelled data pool negatively affects performance through prior probability
shift, which we suggest may explain this performance drop, and that using the
Frechet Distance between labelled and unlabelled data-sets as a measure of
data-set shift can provide a prediction of model performance, but that for
typical radio galaxy data-sets with labelled sample volumes of O(1000), the
sample variance associated with this technique is high and the technique is in
general not sufficiently robust to replace a train-test cycle.
