G-SimCLR : Self-Supervised Contrastive Learning with Guided Projection
  via Pseudo Labelling
  In the realms of computer vision, it is evident that deep neural networks
perform better in a supervised setting with a large amount of labeled data. The
representations learned with supervision are not only of high quality but also
helps the model in enhancing its accuracy. However, the collection and
annotation of a large dataset are costly and time-consuming. To avoid the same,
there has been a lot of research going on in the field of unsupervised visual
representation learning especially in a self-supervised setting. Amongst the
recent advancements in self-supervised methods for visual recognition, in
SimCLR Chen et al. shows that good quality representations can indeed be
learned without explicit supervision. In SimCLR, the authors maximize the
similarity of augmentations of the same image and minimize the similarity of
augmentations of different images. A linear classifier trained with the
representations learned using this approach yields 76.5% top-1 accuracy on the
ImageNet ILSVRC-2012 dataset. In this work, we propose that, with the
normalized temperature-scaled cross-entropy (NT-Xent) loss function (as used in
SimCLR), it is beneficial to not have images of the same category in the same
batch. In an unsupervised setting, the information of images pertaining to the
same category is missing. We use the latent space representation of a denoising
autoencoder trained on the unlabeled dataset and cluster them with k-means to
obtain pseudo labels. With this apriori information we batch images, where no
two images from the same category are to be found. We report comparable
performance enhancements on the CIFAR10 dataset and a subset of the ImageNet
dataset. We refer to our method as G-SimCLR.
