Unsupervised Style and Content Separation by Minimizing Mutual
  Information for Speech Synthesis
  We present a method to generate speech from input text and a style vector
that is extracted from a reference speech signal in an unsupervised manner,
i.e., no style annotation, such as speaker information, is required. Existing
unsupervised methods, during training, generate speech by computing style from
the corresponding ground truth sample and use a decoder to combine the style
vector with the input text. Training the model in such a way leaks content
information into the style vector. The decoder can use the leaked content and
ignore some of the input text to minimize the reconstruction loss. At inference
time, when the reference speech does not match the content input, the output
may not contain all of the content of the input text. We refer to this problem
as "content leakage", which we address by explicitly estimating and minimizing
the mutual information between the style and the content through an adversarial
training formulation. We call our method MIST - Mutual Information based Style
Content Separation. The main goal of the method is to preserve the input
content in the synthesized speech signal, which we measure by the word error
rate (WER) and show substantial improvements over state-of-the-art unsupervised
speech synthesis methods.
