Multi-Format Contrastive Learning of Audio Representations
  Recent advances suggest the advantage of multi-modal training in comparison
with single-modal methods. In contrast to this view, in our work we find that
similar gain can be obtained from training with different formats of a single
modality. In particular, we investigate the use of the contrastive learning
framework to learn audio representations by maximizing the agreement between
the raw audio and its spectral representation. We find a significant gain using
this multi-format strategy against the single-format counterparts. Moreover, on
the downstream AudioSet and ESC-50 classification task, our audio-only approach
achieves new state-of-the-art results with a mean average precision of 0.376
and an accuracy of 90.5%, respectively.
