Scaling and Benchmarking Self-Supervised Visual Representation Learning
  Self-supervised learning aims to learn representations from the data itself
without explicit manual supervision. Existing efforts ignore a crucial aspect
of self-supervised learning - the ability to scale to large amount of data
because self-supervision requires no manual labels. In this work, we revisit
this principle and scale two popular self-supervised approaches to 100 million
images. We show that by scaling on various axes (including data size and
problem 'hardness'), one can largely match or even exceed the performance of
supervised pre-training on a variety of tasks such as object detection, surface
normal estimation (3D) and visual navigation using reinforcement learning.
Scaling these methods also provides many interesting insights into the
limitations of current self-supervised techniques and evaluations. We conclude
that current self-supervised methods are not 'hard' enough to take full
advantage of large scale data and do not seem to learn effective high level
semantic representations. We also introduce an extensive benchmark across 9
different datasets and tasks. We believe that such a benchmark along with
comparable evaluation settings is necessary to make meaningful progress. Code
is at: https://github.com/facebookresearch/fair_self_supervision_benchmark.
