AlphaMatch: Improving Consistency for Semi-supervised Learning with
  Alpha-divergence
  Semi-supervised learning (SSL) is a key approach toward more data-efficient
machine learning by jointly leverage both labeled and unlabeled data. We
propose AlphaMatch, an efficient SSL method that leverages data augmentations,
by efficiently enforcing the label consistency between the data points and the
augmented data derived from them. Our key technical contribution lies on: 1)
using alpha-divergence to prioritize the regularization on data with high
confidence, achieving a similar effect as FixMatch but in a more flexible
fashion, and 2) proposing an optimization-based, EM-like algorithm to enforce
the consistency, which enjoys better convergence than iterative regularization
procedures used in recent SSL methods such as FixMatch, UDA, and MixMatch.
AlphaMatch is simple and easy to implement, and consistently outperforms prior
arts on standard benchmarks, e.g. CIFAR-10, SVHN, CIFAR-100, STL-10.
Specifically, we achieve 91.3% test accuracy on CIFAR-10 with just 4 labelled
data per class, substantially improving over the previously best 88.7% accuracy
achieved by FixMatch.
