A Framework for Generative and Contrastive Learning of Audio
  Representations
  In this paper, we present a framework for contrastive learning for audio
representations, in a self supervised frame work without access to any ground
truth labels. The core idea in self supervised contrastive learning is to map
an audio signal and its various augmented versions (representative of salient
aspects of audio like pitch, timbre etc.) to a space where they are close
together, and are separated from other different signals. In addition we also
explore generative models based on state of the art transformer based
architectures for learning latent spaces for audio signals, without access to
any labels. Here, we map audio signals on a smaller scale to discrete
dictionary elements and train transformers to predict the next dictionary
element. We only use data as a method of supervision, bypassing the need of
labels needed to act as a supervision for training the deep neural networks. We
then use a linear classifier head in order to evaluate the performance of our
models, for both self supervised contrastive and generative transformer based
representations that are learned. Our system achieves considerable performance,
compared to a fully supervised method, with access to ground truth labels to
train the neural network model. These representations, with avail-ability of
large scale audio data show promise in various tasks for audio understanding
tasks
