SynthASR: Unlocking Synthetic Data for Speech Recognition
  End-to-end (E2E) automatic speech recognition (ASR) models have recently
demonstrated superior performance over the traditional hybrid ASR models.
Training an E2E ASR model requires a large amount of data which is not only
expensive but may also raise dependency on production data. At the same time,
synthetic speech generated by the state-of-the-art text-to-speech (TTS) engines
has advanced to near-human naturalness. In this work, we propose to utilize
synthetic speech for ASR training (SynthASR) in applications where data is
sparse or hard to get for ASR model training. In addition, we apply continual
learning with a novel multi-stage training strategy to address catastrophic
forgetting, achieved by a mix of weighted multi-style training, data
augmentation, encoder freezing, and parameter regularization. In our
experiments conducted on in-house datasets for a new application of recognizing
medication names, training ASR RNN-T models with synthetic audio via the
proposed multi-stage training improved the recognition performance on new
application by more than 65% relative, without degradation on existing general
applications. Our observations show that SynthASR holds great promise in
training the state-of-the-art large-scale E2E ASR models for new applications
while reducing the costs and dependency on production data.
