Compositional Attention: Disentangling Search and Retrieval
  Multi-head, key-value attention is the backbone of the widely successful
Transformer model and its variants. This attention mechanism uses multiple
parallel key-value attention blocks (called heads), each performing two
fundamental computations: (1) search - selection of a relevant entity from a
set via query-key interactions, and (2) retrieval - extraction of relevant
features from the selected entity via a value matrix. Importantly, standard
attention heads learn a rigid mapping between search and retrieval. In this
work, we first highlight how this static nature of the pairing can potentially:
(a) lead to learning of redundant parameters in certain tasks, and (b) hinder
generalization. To alleviate this problem, we propose a novel attention
mechanism, called Compositional Attention, that replaces the standard head
structure. The proposed mechanism disentangles search and retrieval and
composes them in a dynamic, flexible and context-dependent manner through an
additional soft competition stage between the query-key combination and value
pairing. Through a series of numerical experiments, we show that it outperforms
standard multi-head attention on a variety of tasks, including some
out-of-distribution settings. Through our qualitative analysis, we demonstrate
that Compositional Attention leads to dynamic specialization based on the type
of retrieval needed. Our proposed mechanism generalizes multi-head attention,
allows independent scaling of search and retrieval, and can easily be
implemented in lieu of standard attention heads in any network architecture.
