Decentralized Unsupervised Learning of Visual Representations
  Collaborative learning enables distributed clients to learn a shared model
for prediction while keeping the training data local on each client. However,
existing collaborative learning methods require fully-labeled data for
training, which is inconvenient or sometimes infeasible to obtain due to the
high labeling cost and the requirement of expertise. The lack of labels makes
collaborative learning impractical in many realistic settings. Self-supervised
learning can address this challenge by learning from unlabeled data.
Contrastive learning (CL), a self-supervised learning approach, can effectively
learn visual representations from unlabeled image data. However, the
distributed data collected on clients are usually not independent and
identically distributed (non-IID) among clients, and each client may only have
few classes of data, which degrades the performance of CL and learned
representations. To tackle this problem, we propose a collaborative contrastive
learning framework consisting of two approaches: feature fusion and
neighborhood matching, by which a unified feature space among clients is
learned for better data representations. Feature fusion provides remote
features as accurate contrastive information to each client for better local
learning. Neighborhood matching further aligns each client's local features to
the remote features such that well-clustered features among clients can be
learned. Extensive experiments show the effectiveness of the proposed
framework. It outperforms other methods by 11% on IID data and matches the
performance of centralized learning.
