Unaligned Supervision For Automatic Music Transcription in The Wild
  Multi-instrument Automatic Music Transcription (AMT), or the decoding of a
musical recording into semantic musical content, is one of the holy grails of
Music Information Retrieval. Current AMT approaches are restricted to piano and
(some) guitar recordings, due to difficult data collection. In order to
overcome data collection barriers, previous AMT approaches attempt to employ
musical scores in the form of a digitized version of the same song or piece.
The scores are typically aligned using audio features and strenuous human
intervention to generate training labels. We introduce NoteEM, a method for
simultaneously training a transcriber and aligning the scores to their
corresponding performances, in a fully-automated process. Using this unaligned
supervision scheme, complemented by pseudo-labels and pitch-shift augmentation,
our method enables training on in-the-wild recordings with unprecedented
accuracy and instrumental variety. Using only synthetic data and unaligned
supervision, we report SOTA note-level accuracy of the MAPS dataset, and large
favorable margins on cross-dataset evaluations. We also demonstrate robustness
and ease of use; we report comparable results when training on a small, easily
obtainable, self-collected dataset, and we propose alternative labeling to the
MusicNet dataset, which we show to be more accurate. Our project page is
available at https://benadar293.github.io
