End-to-End Adversarial Text-to-Speech
  Modern text-to-speech synthesis pipelines typically involve multiple
processing stages, each of which is designed or learnt independently from the
rest. In this work, we take on the challenging task of learning to synthesise
speech from normalised text or phonemes in an end-to-end manner, resulting in
models which operate directly on character or phoneme input sequences and
produce raw speech audio outputs. Our proposed generator is feed-forward and
thus efficient for both training and inference, using a differentiable
alignment scheme based on token length prediction. It learns to produce high
fidelity audio through a combination of adversarial feedback and prediction
losses constraining the generated audio to roughly match the ground truth in
terms of its total duration and mel-spectrogram. To allow the model to capture
temporal variation in the generated audio, we employ soft dynamic time warping
in the spectrogram-based prediction loss. The resulting model achieves a mean
opinion score exceeding 4 on a 5 point scale, which is comparable to the
state-of-the-art models relying on multi-stage training and additional
supervision.
