Hierarchical Multi-Grained Generative Model for Expressive Speech
  Synthesis
  This paper proposes a hierarchical generative model with a multi-grained
latent variable to synthesize expressive speech. In recent years, fine-grained
latent variables are introduced into the text-to-speech synthesis that enable
the fine control of the prosody and speaking styles of synthesized speech.
However, the naturalness of speech degrades when these latent variables are
obtained by sampling from the standard Gaussian prior. To solve this problem,
we propose a novel framework for modeling the fine-grained latent variables,
considering the dependence on an input text, a hierarchical linguistic
structure, and a temporal structure of latent variables. This framework
consists of a multi-grained variational autoencoder, a conditional prior, and a
multi-level auto-regressive latent converter to obtain the different
time-resolution latent variables and sample the finer-level latent variables
from the coarser-level ones by taking into account the input text. Experimental
results indicate an appropriate method of sampling fine-grained latent
variables without the reference signal at the synthesis stage. Our proposed
framework also provides the controllability of speaking style in an entire
utterance.
