AttViz: Online exploration of self-attention for transparent neural
  language modeling
  Neural language models are becoming the prevailing methodology for the tasks
of query answering, text classification, disambiguation, completion and
translation. Commonly comprised of hundreds of millions of parameters, these
neural network models offer state-of-the-art performance at the cost of
interpretability; humans are no longer capable of tracing and understanding how
decisions are being made. The attention mechanism, introduced initially for the
task of translation, has been successfully adopted for other language-related
tasks. We propose AttViz, an online toolkit for exploration of
self-attention---real values associated with individual text tokens. We show
how existing deep learning pipelines can produce outputs suitable for AttViz,
offering novel visualizations of the attention heads and their aggregations
with minimal effort, online. We show on examples of news segments how the
proposed system can be used to inspect and potentially better understand what a
model has learned (or emphasized).
