Fast and Accurate Recurrent Neural Network Acoustic Models for Speech
  Recognition
  We have recently shown that deep Long Short-Term Memory (LSTM) recurrent
neural networks (RNNs) outperform feed forward deep neural networks (DNNs) as
acoustic models for speech recognition. More recently, we have shown that the
performance of sequence trained context dependent (CD) hidden Markov model
(HMM) acoustic models using such LSTM RNNs can be equaled by sequence trained
phone models initialized with connectionist temporal classification (CTC). In
this paper, we present techniques that further improve performance of LSTM RNN
acoustic models for large vocabulary speech recognition. We show that frame
stacking and reduced frame rate lead to more accurate models and faster
decoding. CD phone modeling leads to further improvements. We also present
initial results for LSTM RNN models outputting words directly.
