Contrastive Neural Processes for Self-Supervised Learning
  Recent contrastive methods show significant improvement in self-supervised
learning in several domains. In particular, contrastive methods are most
effective where data augmentation can be easily constructed e.g. in computer
vision. However, they are less successful in domains without established data
transformations such as time series data. In this paper, we propose a novel
self-supervised learning framework that combines contrastive learning with
neural processes. It relies on recent advances in neural processes to perform
time series forecasting. This allows to generate augmented versions of data by
employing a set of various sampling functions and, hence, avoid manually
designed augmentations. We extend conventional neural processes and propose a
new contrastive loss to learn times series representations in a self-supervised
setup. Therefore, unlike previous self-supervised methods, our augmentation
pipeline is task-agnostic, enabling our method to perform well across various
applications. In particular, a ResNet with a linear classifier trained using
our approach is able to outperform state-of-the-art techniques across
industrial, medical and audio datasets improving accuracy over 10% in ECG
periodic data. We further demonstrate that our self-supervised representations
are more efficient in the latent space, improving multiple clustering indexes
and that fine-tuning our method on 10% of labels achieves results competitive
to fully-supervised learning.
