Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognition
  With recent advances in speech synthesis, synthetic data is becoming a viable
alternative to real data for training speech recognition models. However,
machine learning with synthetic data is not trivial due to the gap between the
synthetic and the real data distributions. Synthetic datasets may contain
artifacts that do not exist in real data such as structured noise, content
errors, or unrealistic speaking styles. Moreover, the synthesis process may
introduce a bias due to uneven sampling of the data manifold. We propose two
novel techniques during training to mitigate the problems due to the
distribution gap: (i) a rejection sampling algorithm and (ii) using separate
batch normalization statistics for the real and the synthetic samples. We show
that these methods significantly improve the training of speech recognition
models using synthetic data. We evaluate the proposed approach on keyword
detection and Automatic Speech Recognition (ASR) tasks, and observe up to 18%
and 13% relative error reduction, respectively, compared to naively using the
synthetic data.
