Hypernetworks for Continual Semi-Supervised Learning
  Learning from data sequentially arriving, possibly in a non i.i.d. way, with
changing task distribution over time is called continual learning. Much of the
work thus far in continual learning focuses on supervised learning and some
recent works on unsupervised learning. In many domains, each task contains a
mix of labelled (typically very few) and unlabelled (typically plenty) training
examples, which necessitates a semi-supervised learning approach. To address
this in a continual learning setting, we propose a framework for
semi-supervised continual learning called Meta-Consolidation for Continual
Semi-Supervised Learning (MCSSL). Our framework has a hypernetwork that learns
the meta-distribution that generates the weights of a semi-supervised auxiliary
classifier generative adversarial network $(\textit{Semi-ACGAN})$ as the base
network. We consolidate the knowledge of sequential tasks in the hypernetwork,
and the base network learns the semi-supervised learning task. Further, we
present $\textit{Semi-Split CIFAR-10}$, a new benchmark for continual
semi-supervised learning, obtained by modifying the $\textit{Split CIFAR-10}$
dataset, in which the tasks with labelled and unlabelled data arrive
sequentially. Our proposed model yields significant improvements in the
continual semi-supervised learning setting. We compare the performance of
several existing continual learning approaches on the proposed continual
semi-supervised learning benchmark of the Semi-Split CIFAR-10 dataset.
