Sinsy: A Deep Neural Network-Based Singing Voice Synthesis System
  This paper presents Sinsy, a deep neural network (DNN)-based singing voice
synthesis (SVS) system. In recent years, DNNs have been utilized in statistical
parametric SVS systems, and DNN-based SVS systems have demonstrated better
performance than conventional hidden Markov model-based ones. SVS systems are
required to synthesize a singing voice with pitch and timing that strictly
follow a given musical score. Additionally, singing expressions that are not
described on the musical score, such as vibrato and timing fluctuations, should
be reproduced. The proposed system is composed of four modules: a time-lag
model, a duration model, an acoustic model, and a vocoder, and singing voices
can be synthesized taking these characteristics of singing voices into account.
To better model a singing voice, the proposed system incorporates improved
approaches to modeling pitch and vibrato and better training criteria into the
acoustic model. In addition, we incorporated PeriodNet, a non-autoregressive
neural vocoder with robustness for the pitch, into our systems to generate a
high-fidelity singing voice waveform. Moreover, we propose automatic pitch
correction techniques for DNN-based SVS to synthesize singing voices with
correct pitch even if the training data has out-of-tune phrases. Experimental
results show our system can synthesize a singing voice with better timing, more
natural vibrato, and correct pitch, and it can achieve better mean opinion
scores in subjective evaluation tests.
