Visual Reasoning of Feature Attribution with Deep Recurrent Neural
  Networks
  Deep Recurrent Neural Network (RNN) has gained popularity in many sequence
classification tasks. Beyond predicting a correct class for each data instance,
data scientists also want to understand what differentiating factors in the
data have contributed to the classification during the learning process. We
present a visual analytics approach to facilitate this task by revealing the
RNN attention for all data instances, their temporal positions in the
sequences, and the attribution of variables at each value level. We demonstrate
with real-world datasets that our approach can help data scientists to
understand such dynamics in deep RNNs from the training results, hence guiding
their modeling process.
