Learning the Beauty in Songs: Neural Singing Voice Beautifier
  We are interested in a novel task, singing voice beautifying (SVB). Given the
singing voice of an amateur singer, SVB aims to improve the intonation and
vocal tone of the voice, while keeping the content and vocal timbre. Current
automatic pitch correction techniques are immature, and most of them are
restricted to intonation but ignore the overall aesthetic quality. Hence, we
introduce Neural Singing Voice Beautifier (NSVB), the first generative model to
solve the SVB task, which adopts a conditional variational autoencoder as the
backbone and learns the latent representations of vocal tone. In NSVB, we
propose a novel time-warping approach for pitch correction: Shape-Aware Dynamic
Time Warping (SADTW), which ameliorates the robustness of existing time-warping
approaches, to synchronize the amateur recording with the template pitch curve.
Furthermore, we propose a latent-mapping algorithm in the latent space to
convert the amateur vocal tone to the professional one. To achieve this, we
also propose a new dataset containing parallel singing recordings of both
amateur and professional versions. Extensive experiments on both Chinese and
English songs demonstrate the effectiveness of our methods in terms of both
objective and subjective metrics. Audio samples are available
at~\url{https://neuralsvb.github.io}. Codes:
\url{https://github.com/MoonInTheRiver/NeuralSVB}.
