Increasing the Interpretability of Recurrent Neural Networks Using
  Hidden Markov Models
  As deep neural networks continue to revolutionize various application
domains, there is increasing interest in making these powerful models more
understandable and interpretable, and narrowing down the causes of good and bad
predictions. We focus on recurrent neural networks (RNNs), state of the art
models in speech recognition and translation. Our approach to increasing
interpretability is by combining an RNN with a hidden Markov model (HMM), a
simpler and more transparent model. We explore various combinations of RNNs and
HMMs: an HMM trained on LSTM states; a hybrid model where an HMM is trained
first, then a small LSTM is given HMM state distributions and trained to fill
in gaps in the HMM's performance; and a jointly trained hybrid model. We find
that the LSTM and HMM learn complementary information about the features in the
text.
