Building Interpretable Interaction Trees for Deep NLP Models
  This paper proposes a method to disentangle and quantify interactions among
words that are encoded inside a DNN for natural language processing. We
construct a tree to encode salient interactions extracted by the DNN. Six
metrics are proposed to analyze properties of interactions between constituents
in a sentence. The interaction is defined based on Shapley values of words,
which are considered as an unbiased estimation of word contributions to the
network prediction. Our method is used to quantify word interactions encoded
inside the BERT, ELMo, LSTM, CNN, and Transformer networks. Experimental
results have provided a new perspective to understand these DNNs, and have
demonstrated the effectiveness of our method.
