Streaming Self-Training via Domain-Agnostic Unlabeled Images
  We present streaming self-training (SST) that aims to democratize the process
of learning visual recognition models such that a non-expert user can define a
new task depending on their needs via a few labeled examples and minimal domain
knowledge. Key to SST are two crucial observations: (1) domain-agnostic
unlabeled images enable us to learn better models with a few labeled examples
without any additional knowledge or supervision; and (2) learning is a
continuous process and can be done by constructing a schedule of learning
updates that iterates between pre-training on novel segments of the streams of
unlabeled data, and fine-tuning on the small and fixed labeled dataset. This
allows SST to overcome the need for a large number of domain-specific labeled
and unlabeled examples, exorbitant computational resources, and
domain/task-specific knowledge. In this setting, classical semi-supervised
approaches require a large amount of domain-specific labeled and unlabeled
examples, immense resources to process data, and expert knowledge of a
particular task. Due to these reasons, semi-supervised learning has been
restricted to a few places that can house required computational and human
resources. In this work, we overcome these challenges and demonstrate our
findings for a wide range of visual recognition tasks including fine-grained
image classification, surface normal estimation, and semantic segmentation. We
also demonstrate our findings for diverse domains including medical, satellite,
and agricultural imagery, where there does not exist a large amount of labeled
or unlabeled data.
