Self-Supervised Learning from Automatically Separated Sound Scenes
  Real-world sound scenes consist of time-varying collections of sound sources,
each generating characteristic sound events that are mixed together in audio
recordings. The association of these constituent sound events with their
mixture and each other is semantically constrained: the sound scene contains
the union of source classes and not all classes naturally co-occur. With this
motivation, this paper explores the use of unsupervised automatic sound
separation to decompose unlabeled sound scenes into multiple
semantically-linked views for use in self-supervised contrastive learning. We
find that learning to associate input mixtures with their automatically
separated outputs yields stronger representations than past approaches that use
the mixtures alone. Further, we discover that optimal source separation is not
required for successful contrastive learning by demonstrating that a range of
separation system convergence states all lead to useful and often complementary
example transformations. Our best system incorporates these unsupervised
separation models into a single augmentation front-end and jointly optimizes
similarity maximization and coincidence prediction objectives across the views.
The result is an unsupervised audio representation that rivals state-of-the-art
alternatives on the established shallow AudioSet classification benchmark.
