Contrastive learning of strong-mixing continuous-time stochastic
  processes
  Contrastive learning is a family of self-supervised methods where a model is
trained to solve a classification task constructed from unlabeled data. It has
recently emerged as one of the leading learning paradigms in the absence of
labels across many different domains (e.g. brain imaging, text, images).
However, theoretical understanding of many aspects of training, both
statistical and algorithmic, remain fairly elusive.
  In this work, we study the setting of time series -- more precisely, when we
get data from a strong-mixing continuous-time stochastic process. We show that
a properly constructed contrastive learning task can be used to estimate the
transition kernel for small-to-mid-range intervals in the diffusion case.
Moreover, we give sample complexity bounds for solving this task and
quantitatively characterize what the value of the contrastive loss implies for
distributional closeness of the learned kernel. As a byproduct, we illuminate
the appropriate settings for the contrastive distribution, as well as other
hyperparameters in this setup.
