Unsupervised Cross-Modal Audio Representation Learning from Unstructured
  Multilingual Text
  We present an approach to unsupervised audio representation learning. Based
on a triplet neural network architecture, we harnesses semantically related
cross-modal information to estimate audio track-relatedness. By applying Latent
Semantic Indexing (LSI) we embed corresponding textual information into a
latent vector space from which we derive track relatedness for online triplet
selection. This LSI topic modelling facilitates fine-grained selection of
similar and dissimilar audio-track pairs to learn the audio representation
using a Convolution Recurrent Neural Network (CRNN). By this we directly
project the semantic context of the unstructured text modality onto the learned
representation space of the audio modality without deriving structured
ground-truth annotations from it. We evaluate our approach on the Europeana
Sounds collection and show how to improve search in digital audio libraries by
harnessing the multilingual meta-data provided by numerous European digital
libraries. We show that our approach is invariant to the variety of annotation
styles as well as to the different languages of this collection. The learned
representations perform comparable to the baseline of handcrafted features,
respectively exceeding this baseline in similarity retrieval precision at
higher cut-offs with only 15\% of the baseline's feature vector length.
