Assem-VC: Realistic Voice Conversion by Assembling Modern Speech
  Synthesis Techniques
  Recent works on voice conversion (VC) focus on preserving the rhythm and the
intonation as well as the linguistic content. To preserve these features from
the source, we decompose current non-parallel VC systems into two encoders and
one decoder. We analyze each module with several experiments and reassemble the
best components to propose Assem-VC, a new state-of-the-art any-to-many
non-parallel VC system. We also examine that PPG and Cotatron features are
speaker-dependent, and attempt to remove speaker identity with adversarial
training. Code and audio samples are available at
https://github.com/mindslab-ai/assem-vc.
