Survey on Self-supervised Representation Learning Using Image
  Transformations
  Deep neural networks need huge amount of training data, while in real world
there is a scarcity of data available for training purposes. To resolve these
issues, self-supervised learning (SSL) methods are used. SSL using geometric
transformations (GT) is a simple yet powerful technique used in unsupervised
representation learning. Although multiple survey papers have reviewed SSL
techniques, there is none that only focuses on those that use geometric
transformations. Furthermore, such methods have not been covered in depth in
papers where they are reviewed. Our motivation to present this work is that
geometric transformations have shown to be powerful supervisory signals in
unsupervised representation learning. Moreover, many such works have found
tremendous success, but have not gained much attention. We present a concise
survey of SSL approaches that use geometric transformations. We shortlist six
representative models that use image transformations including those based on
predicting and autoencoding transformations. We review their architecture as
well as learning methodologies. We also compare the performance of these models
in the object recognition task on CIFAR-10 and ImageNet datasets. Our analysis
indicates the AETv2 performs the best in most settings. Rotation with feature
decoupling also performed well in some settings. We then derive insights from
the observed results. Finally, we conclude with a summary of the results and
insights as well as highlighting open problems to be addressed and indicating
various future directions.
