Visualizing Attention in Transformer-Based Language Representation
  Models
  We present an open-source tool for visualizing multi-head self-attention in
Transformer-based language representation models. The tool extends earlier work
by visualizing attention at three levels of granularity: the attention-head
level, the model level, and the neuron level. We describe how each of these
views can help to interpret the model, and we demonstrate the tool on the BERT
model and the OpenAI GPT-2 model. We also present three use cases for analyzing
GPT-2: detecting model bias, identifying recurring patterns, and linking
neurons to model behavior.
