Vocoder-free End-to-End Voice Conversion with Transformer Network
  Mel-frequency filter bank (MFB) based approaches have the advantage of
learning speech compared to raw spectrum since MFB has less feature size.
However, speech generator with MFB approaches require additional vocoder that
needs a huge amount of computation expense for training process. The additional
pre/post processing such as MFB and vocoder is not essential to convert real
human speech to others. It is possible to only use the raw spectrum along with
the phase to generate different style of voices with clear pronunciation. In
this regard, we propose a fast and effective approach to convert realistic
voices using raw spectrum in a parallel manner. Our transformer-based model
architecture which does not have any CNN or RNN layers has shown the advantage
of learning fast and solved the limitation of sequential computation of
conventional RNN. In this paper, we introduce a vocoder-free end-to-end voice
conversion method using transformer network. The presented conversion model can
also be used in speaker adaptation for speech recognition. Our approach can
convert the source voice to a target voice without using MFB and vocoder. We
can get an adapted MFB for speech recognition by multiplying the converted
magnitude with phase. We perform our voice conversion experiments on TIDIGITS
dataset using the metrics such as naturalness, similarity, and clarity with
mean opinion score, respectively.
