CTC-synchronous Training for Monotonic Attention Model
  Monotonic chunkwise attention (MoChA) has been studied for the online
streaming automatic speech recognition (ASR) based on a sequence-to-sequence
framework. In contrast to connectionist temporal classification (CTC), backward
probabilities cannot be leveraged in the alignment marginalization process
during training due to left-to-right dependency in the decoder. This results in
the error propagation of alignments to subsequent token generation. To address
this problem, we propose CTC-synchronous training (CTC-ST), in which MoChA uses
CTC alignments to learn optimal monotonic alignments. Reference CTC alignments
are extracted from a CTC branch sharing the same encoder with the decoder. The
entire model is jointly optimized so that the expected boundaries from MoChA
are synchronized with the alignments. Experimental evaluations of the TEDLIUM
release-2 and Librispeech corpora show that the proposed method significantly
improves recognition, especially for long utterances. We also show that CTC-ST
can bring out the full potential of SpecAugment for MoChA.
