CLAR: Contrastive Learning of Auditory Representations
  Learning rich visual representations using contrastive self-supervised
learning has been extremely successful. However, it is still a major question
whether we could use a similar approach to learn superior auditory
representations. In this paper, we expand on prior work (SimCLR) to learn
better auditory representations. We (1) introduce various data augmentations
suitable for auditory data and evaluate their impact on predictive performance,
(2) show that training with time-frequency audio features substantially
improves the quality of the learned representations compared to raw signals,
and (3) demonstrate that training with both supervised and contrastive losses
simultaneously improves the learned representations compared to self-supervised
pre-training followed by supervised fine-tuning. We illustrate that by
combining all these methods and with substantially less labeled data, our
framework (CLAR) achieves significant improvement on prediction performance
compared to supervised approach. Moreover, compared to self-supervised
approach, our framework converges faster with significantly better
representations.
