Explaining the Attention Mechanism of End-to-End Speech Recognition
  Using Decision Trees
  The attention mechanism has largely improved the performance of end-to-end
speech recognition systems. However, the underlying behaviours of attention is
not yet clearer. In this study, we use decision trees to explain how the
attention mechanism impact itself in speech recognition. The results indicate
that attention levels are largely impacted by their previous states rather than
the encoder and decoder patterns. Additionally, the default attention mechanism
seems to put more weights on closer states, but behaves poorly on modelling
long-term dependencies of attention states.
