Attention based on-device streaming speech recognition with large speech
  corpus
  In this paper, we present a new on-device automatic speech recognition (ASR)
system based on monotonic chunk-wise attention (MoChA) models trained with
large (> 10K hours) corpus. We attained around 90% of a word recognition rate
for general domain mainly by using joint training of connectionist temporal
classifier (CTC) and cross entropy (CE) losses, minimum word error rate (MWER)
training, layer-wise pre-training and data augmentation methods. In addition,
we compressed our models by more than 3.4 times smaller using an iterative
hyper low-rank approximation (LRA) method while minimizing the degradation in
recognition accuracy. The memory footprint was further reduced with 8-bit
quantization to bring down the final model size to lower than 39 MB. For
on-demand adaptation, we fused the MoChA models with statistical n-gram models,
and we could achieve a relatively 36% improvement on average in word error rate
(WER) for target domains including the general domain.
