A Lightweight Instrument-Agnostic Model for Polyphonic Note
  Transcription and Multipitch Estimation
  Automatic Music Transcription (AMT) has been recognized as a key enabling
technology with a wide range of applications. Given the task's complexity, best
results have typically been reported for systems focusing on specific settings,
e.g. instrument-specific systems tend to yield improved results over
instrument-agnostic methods. Similarly, higher accuracy can be obtained when
only estimating frame-wise $f_0$ values and neglecting the harder note event
detection. Despite their high accuracy, such specialized systems often cannot
be deployed in the real-world. Storage and network constraints prohibit the use
of multiple specialized models, while memory and run-time constraints limit
their complexity. In this paper, we propose a lightweight neural network for
musical instrument transcription, which supports polyphonic outputs and
generalizes to a wide variety of instruments (including vocals). Our model is
trained to jointly predict frame-wise onsets, multipitch and note activations,
and we experimentally show that this multi-output structure improves the
resulting frame-level note accuracy. Despite its simplicity, benchmark results
show our system's note estimation to be substantially better than a comparable
baseline, and its frame-level accuracy to be only marginally below those of
specialized state-of-the-art AMT systems. With this work we hope to encourage
the community to further investigate low-resource, instrument-agnostic AMT
systems.
