Delving into Inter-Image Invariance for Unsupervised Visual
  Representations
  Contrastive learning has recently shown immense potential in unsupervised
visual representation learning. Existing studies in this track mainly focus on
intra-image invariance learning. The learning typically uses rich intra-image
transformations to construct positive pairs and then maximizes agreement using
a contrastive loss. The merits of inter-image invariance, conversely, remain
much less explored. One major obstacle to exploit inter-image invariance is
that it is unclear how to reliably construct inter-image positive pairs, and
further derive effective supervision from them since there are no pair
annotations available. In this work, we present a rigorous and comprehensive
study on inter-image invariance learning from three main constituting
components: pseudo-label maintenance, sampling strategy, and decision boundary
design. Through carefully-designed comparisons and analysis, we propose a
unified and generic framework that supports the integration of unsupervised
intra- and inter-image invariance learning. With all the obtained recipes, our
final model, namely InterCLR, shows consistent improvements over
state-of-the-art intra-image invariance learning methods on multiple standard
benchmarks. Codes will be released at
https://github.com/open-mmlab/OpenSelfSup.
