Differentiable Digital Signal Processing Mixture Model for Synthesis
  Parameter Extraction from Mixture of Harmonic Sounds
  A differentiable digital signal processing (DDSP) autoencoder is a musical
sound synthesizer that combines a deep neural network (DNN) and spectral
modeling synthesis. It allows us to flexibly edit sounds by changing the
fundamental frequency, timbre feature, and loudness (synthesis parameters)
extracted from an input sound. However, it is designed for a monophonic
harmonic sound and cannot handle mixtures of harmonic sounds. In this paper, we
propose a model (DDSP mixture model) that represents a mixture as the sum of
the outputs of multiple pretrained DDSP autoencoders. By fitting the output of
the proposed model to the observed mixture, we can directly estimate the
synthesis parameters of each source. Through synthesis parameter extraction
experiments, we show that the proposed method has high and stable performance
compared with a straightforward method that applies the DDSP autoencoder to the
signals separated by an audio source separation method.
