Contrastive learning, multi-view redundancy, and linear models
  Self-supervised learning is an empirically successful approach to
unsupervised learning based on creating artificial supervised learning
problems. A popular self-supervised approach to representation learning is
contrastive learning, which leverages naturally occurring pairs of similar and
dissimilar data points, or multiple views of the same data. This work provides
a theoretical analysis of contrastive learning in the multi-view setting, where
two views of each datum are available. The main result is that linear functions
of the learned representations are nearly optimal on downstream prediction
tasks whenever the two views provide redundant information about the label.
