Semi-Supervised Learning with Multi-Head Co-Training
  Co-training, extended from self-training, is one of the frameworks for
semi-supervised learning. Without natural split of features, single-view
co-training works at the cost of training extra classifiers, where the
algorithm should be delicately designed to prevent individual classifiers from
collapsing into each other. To remove these obstacles which deter the adoption
of single-view co-training, we present a simple and efficient algorithm
Multi-Head Co-Training. By integrating base learners into a multi-head
structure, the model is in a minimal amount of extra parameters. Every
classification head in the unified model interacts with its peers through a
"Weak and Strong Augmentation" strategy, in which the diversity is naturally
brought by the strong data augmentation. Therefore, the proposed method
facilitates single-view co-training by 1). promoting diversity implicitly and
2). only requiring a small extra computational overhead. The effectiveness of
Multi-Head Co-Training is demonstrated in an empirical study on standard
semi-supervised learning benchmarks.
