Data Transformation Insights in Self-supervision with Clustering Tasks
  Self-supervision is key to extending use of deep learning for label scarce
domains. For most of self-supervised approaches data transformations play an
important role. However, up until now the impact of transformations have not
been studied. Furthermore, different transformations may have different impact
on the system. We provide novel insights into the use of data transformation in
self-supervised tasks, specially pertaining to clustering. We show
theoretically and empirically that certain set of transformations are helpful
in convergence of self-supervised clustering. We also show the cases when the
transformations are not helpful or in some cases even harmful. We show faster
convergence rate with valid transformations for convex as well as certain
family of non-convex objectives along with the proof of convergence to the
original set of optima. We have synthetic as well as real world data
experiments. Empirically our results conform with the theoretical insights
provided.
