Decoupled Contrastive Learning
  Contrastive learning (CL) is one of the most successful paradigms for
self-supervised learning (SSL). In a principled way, it considers two augmented
"views" of the same image as positive to be pulled closer, and all other images
negative to be pushed further apart. However, behind the impressive success of
CL-based techniques, their formulation often relies on heavy-computation
settings, including large sample batches, extensive training epochs, etc. We
are thus motivated to tackle these issues and aim at establishing a simple,
efficient, and yet competitive baseline of contrastive learning. Specifically,
we identify, from theoretical and empirical studies, a noticeable
negative-positive-coupling (NPC) effect in the widely used cross-entropy
(InfoNCE) loss, leading to unsuitable learning efficiency with respect to the
batch size. Indeed the phenomenon tends to be neglected in that optimizing
infoNCE loss with a small-size batch is effective in solving easier SSL tasks.
By properly addressing the NPC effect, we reach a decoupled contrastive
learning (DCL) objective function, significantly improving SSL efficiency. DCL
can achieve competitive performance, requiring neither large batches in SimCLR,
momentum encoding in MoCo, or large epochs. We demonstrate the usefulness of
DCL in various benchmarks, while manifesting its robustness being much less
sensitive to suboptimal hyperparameters. Notably, our approach achieves
$66.9\%$ ImageNet top-1 accuracy using batch size 256 within 200 epochs
pre-training, outperforming its baseline SimCLR by $5.1\%$. With further
optimized hyperparameters, DCL can improve the accuracy to $68.2\%$. We believe
DCL provides a valuable baseline for future contrastive learning-based SSL
studies.
