Evaluating Off-the-Shelf Machine Listening and Natural Language Models
  for Automated Audio Captioning
  Automated audio captioning (AAC) is the task of automatically generating
textual descriptions for general audio signals. A captioning system has to
identify various information from the input signal and express it with natural
language. Existing works mainly focus on investigating new methods and try to
improve their performance measured on existing datasets. Having attracted
attention only recently, very few works on AAC study the performance of
existing pre-trained audio and natural language processing resources. In this
paper, we evaluate the performance of off-the-shelf models with a
Transformer-based captioning approach. We utilize the freely available Clotho
dataset to compare four different pre-trained machine listening models, four
word embedding models, and their combinations in many different settings. Our
evaluation suggests that YAMNet combined with BERT embeddings produces the best
captions. Moreover, in general, fine-tuning pre-trained word embeddings can
lead to better performance. Finally, we show that sequences of audio embeddings
can be processed using a Transformer encoder to produce higher-quality
captions.
