Semi-Unsupervised Learning: Clustering and Classifying using
  Ultra-Sparse Labels
  In semi-supervised learning for classification, it is assumed that every
ground truth class of data is present in the small labelled dataset. Many
real-world sparsely-labelled datasets are plausibly not of this type. It could
easily be the case that some classes of data are found only in the unlabelled
dataset -- perhaps the labelling process was biased -- so we do not have any
labelled examples to train on for some classes. We call this learning regime
$\textit{semi-unsupervised learning}$, an extreme case of semi-supervised
learning, where some classes have no labelled exemplars in the training set.
First, we outline the pitfalls associated with trying to apply deep generative
model (DGM)-based semi-supervised learning algorithms to datasets of this type.
We then show how a combination of clustering and semi-supervised learning,
using DGMs, can be brought to bear on this problem. We study several different
datasets, showing how one can still learn effectively when half of the ground
truth classes are entirely unlabelled and the other half are sparsely labelled.
