Contrastive Learning of Musical Representations
  While deep learning has enabled great advances in many areas of music,
labeled music datasets remain especially hard, expensive, and time-consuming to
create. In this work, we introduce SimCLR to the music domain and contribute a
large chain of audio data augmentations to form a simple framework for
self-supervised, contrastive learning of musical representations: CLMR. This
approach works on raw time-domain music data and requires no labels to learn
useful representations. We evaluate CLMR in the downstream task of music
classification on the MagnaTagATune and Million Song datasets and present an
ablation study to test which of our music-related innovations over SimCLR are
most effective. A linear classifier trained on the proposed representations
achieves a higher average precision than supervised models on the MagnaTagATune
dataset, and performs comparably on the Million Song dataset. Moreover, we show
that CLMR's representations are transferable using out-of-domain datasets,
indicating that our method has strong generalisability in music classification.
Lastly, we show that the proposed method allows data-efficient learning on
smaller labeled datasets: we achieve an average precision of 33.1% despite
using only 259 labeled songs in the MagnaTagATune dataset (1% of the full
dataset) during linear evaluation. To foster reproducibility and future
research on self-supervised learning in music, we publicly release the
pre-trained models and the source code of all experiments of this paper.
