Self Training with Ensemble of Teacher Models
  In order to train robust deep learning models, large amounts of labelled data
is required. However, in the absence of such large repositories of labelled
data, unlabeled data can be exploited for the same. Semi-Supervised learning
aims to utilize such unlabeled data for training classification models. Recent
progress of self-training based approaches have shown promise in this area,
which leads to this study where we utilize an ensemble approach for the same. A
by-product of any semi-supervised approach may be loss of calibration of the
trained model especially in scenarios where unlabeled data may contain
out-of-distribution samples, which leads to this investigation on how to adapt
to such effects. Our proposed algorithm carefully avoids common pitfalls in
utilizing unlabeled data and leads to a more accurate and calibrated supervised
model compared to vanilla self-training based student-teacher algorithms. We
perform several experiments on the popular STL-10 database followed by an
extensive analysis of our approach and study its effects on model accuracy and
calibration.
