Content based singing voice source separation via strong conditioning
  using aligned phonemes
  Informed source separation has recently gained renewed interest with the
introduction of neural networks and the availability of large multitrack
datasets containing both the mixture and the separated sources. These
approaches use prior information about the target source to improve separation.
Historically, Music Information Retrieval researchers have focused primarily on
score-informed source separation, but more recent approaches explore
lyrics-informed source separation. However, because of the lack of multitrack
datasets with time-aligned lyrics, models use weak conditioning with
non-aligned lyrics. In this paper, we present a multimodal multitrack dataset
with lyrics aligned in time at the word level with phonetic information as well
as explore strong conditioning using the aligned phonemes. Our model follows a
U-Net architecture and takes as input both the magnitude spectrogram of a
musical mixture and a matrix with aligned phonetic information. The phoneme
matrix is embedded to obtain the parameters that control Feature-wise Linear
Modulation (FiLM) layers. These layers condition the U-Net feature maps to
adapt the separation process to the presence of different phonemes via affine
transformations. We show that phoneme conditioning can be successfully applied
to improve singing voice source separation.
