HEAR: Holistic Evaluation of Audio Representations
  What audio embedding approach generalizes best to a wide range of downstream
tasks across a variety of everyday domains without fine-tuning? The aim of the
HEAR benchmark is to develop a general-purpose audio representation that
provides a strong basis for learning in a wide variety of tasks and scenarios.
HEAR evaluates audio representations using a benchmark suite across a variety
of domains, including speech, environmental sound, and music. HEAR was launched
as a NeurIPS 2021 shared challenge. In the spirit of shared exchange, each
participant submitted an audio embedding model following a common API that is
general-purpose, open-source, and freely available to use. Twenty-nine models
by thirteen external teams were evaluated on nineteen diverse downstream tasks
derived from sixteen datasets. Open evaluation code, submitted models and
datasets are key contributions, enabling comprehensive and reproducible
evaluation, as well as previously impossible longitudinal studies. It still
remains an open question whether one single general-purpose audio
representation can perform as holistically as the human ear.
