Modeling Document Interactions for Learning to Rank with Regularized
  Self-Attention
  Learning to rank is an important task that has been successfully deployed in
many real-world information retrieval systems. Most existing methods compute
relevance judgments of documents independently, without holistically
considering the entire set of competing documents. In this paper, we explore
modeling documents interactions with self-attention based neural networks.
Although self-attention networks have achieved state-of-the-art results in many
NLP tasks, we find empirically that self-attention provides little benefit over
baseline neural learning to rank architecture. To improve the learning of
self-attention weights, We propose simple yet effective regularization terms
designed to model interactions between documents. Evaluations on publicly
available Learning to Rank (LETOR) datasets show that training self-attention
network with our proposed regularization terms can significantly outperform
existing learning to rank methods.
