Semi-supervised Learning with Contrastive Predicative Coding
  Semi-supervised learning (SSL) provides a powerful framework for leveraging
unlabeled data when labels are limited or expensive to obtain. SSL algorithms
based on deep neural networks have recently proven successful on standard
benchmark tasks. However, many of them have thus far been either inflexible,
inefficient or non-scalable. This paper explores recently developed contrastive
predictive coding technique to improve discriminative power of deep learning
models when a large portion of labels are absent. Two models, cpc-SSL and a
class conditional variant~(ccpc-SSL) are presented. They effectively exploit
the unlabeled data by extracting shared information between different parts of
the (high-dimensional) data. The proposed approaches are inductive, and scale
well to very large datasets like ImageNet, making them good candidates in
real-world large scale applications.
