Cross-modal supervised learning for better acoustic representations
  Obtaining large-scale human-labeled datasets to train acoustic representation
models is a very challenging task. On the contrary, we can easily collect data
with machine-generated labels. In this work, we propose to exploit
machine-generated labels to learn better acoustic representations, based on the
synchronization between vision and audio. Firstly, we collect a large-scale
video dataset with 15 million samples, which totally last 16,320 hours. Each
video is 3 to 5 seconds in length and annotated automatically by publicly
available visual and audio classification models. Secondly, we train various
classical convolutional neural networks (CNNs) including VGGish, ResNet 50 and
Mobilenet v2. We also make several improvements to VGGish and achieve better
results. Finally, we transfer our models on three external standard benchmarks
for audio classification task, and achieve significant performance boost over
the state-of-the-art results. Models and codes are available at:
https://github.com/Deeperjia/vgg-like-audio-models.
