Transfer Learning Approaches for Streaming End-to-End Speech Recognition
  System
  Transfer learning (TL) is widely used in conventional hybrid automatic speech
recognition (ASR) system, to transfer the knowledge from source to target
language. TL can be applied to end-to-end (E2E) ASR system such as recurrent
neural network transducer (RNN-T) models, by initializing the encoder and/or
prediction network of the target language with the pre-trained models from
source language. In the hybrid ASR system, transfer learning is typically done
by initializing the target language acoustic model (AM) with source language
AM. Several transfer learning strategies exist in the case of the RNN-T
framework, depending upon the choice of the initialization model for encoder
and prediction networks. This paper presents a comparative study of four
different TL methods for RNN-T framework. We show 17% relative word error rate
reduction with different TL methods over randomly initialized RNN-T model. We
also study the impact of TL with varying amount of training data ranging from
50 hours to 1000 hours and show the efficacy of TL for languages with small
amount of training data.
