LoCo: Local Contrastive Representation Learning
  Deep neural nets typically perform end-to-end backpropagation to learn the
weights, a procedure that creates synchronization constraints in the weight
update step across layers and is not biologically plausible. Recent advances in
unsupervised contrastive representation learning point to the question of
whether a learning algorithm can also be made local, that is, the updates of
lower layers do not directly depend on the computation of upper layers. While
Greedy InfoMax separately learns each block with a local objective, we found
that it consistently hurts readout accuracy in state-of-the-art unsupervised
contrastive learning algorithms, possibly due to the greedy objective as well
as gradient isolation. In this work, we discover that by overlapping local
blocks stacking on top of each other, we effectively increase the decoder depth
and allow upper blocks to implicitly send feedbacks to lower blocks. This
simple design closes the performance gap between local learning and end-to-end
contrastive learning algorithms for the first time. Aside from standard
ImageNet experiments, we also show results on complex downstream tasks such as
object detection and instance segmentation directly using readout features.
