SelfAugment: Automatic Augmentation Policies for Self-Supervised
  Learning
  A common practice in unsupervised representation learning is to use labeled
data to evaluate the quality of the learned representations. This supervised
evaluation is then used to guide critical aspects of the training process such
as selecting the data augmentation policy. However, guiding an unsupervised
training process through supervised evaluations is not possible for real-world
data that does not actually contain labels (which may be the case, for example,
in privacy sensitive fields such as medical imaging). Therefore, in this work
we show that evaluating the learned representations with a self-supervised
image rotation task is highly correlated with a standard set of supervised
evaluations (rank correlation $> 0.94$). We establish this correlation across
hundreds of augmentation policies, training settings, and network architectures
and provide an algorithm (SelfAugment) to automatically and efficiently select
augmentation policies without using supervised evaluations. Despite not using
any labeled data, the learned augmentation policies perform comparably with
augmentation policies that were determined using exhaustive supervised
evaluations.
