Semi-supervised learning objectives as log-likelihoods in a generative
  model of data curation
  We currently do not have an understanding of semi-supervised learning (SSL)
objectives such as pseudo-labelling and entropy minimization as
log-likelihoods, which precludes the development of e.g. Bayesian SSL. Here, we
note that benchmark image datasets such as CIFAR-10 are carefully curated, and
we formulate SSL objectives as a log-likelihood in a generative model of data
curation that was initially developed to explain the cold-posterior effect
(Aitchison 2020). SSL methods, from entropy minimization and pseudo-labelling,
to state-of-the-art techniques similar to FixMatch can be understood as
lower-bounds on our principled log-likelihood. We are thus able to give a
proof-of-principle for Bayesian SSL on toy data. Finally, our theory suggests
that SSL is effective in part due to the statistical patterns induced by data
curation. This provides an explanation of past results which show SSL performs
better on clean datasets without any "out of distribution" examples. Confirming
these results we find that SSL gave much larger performance improvements on
curated than on uncurated data, using matched curated and uncurated datasets
based on Galaxy Zoo 2.
