Rationalizing Text Matching: Learning Sparse Alignments via Optimal
  Transport
  Selecting input features of top relevance has become a popular method for
building self-explaining models. In this work, we extend this selective
rationalization approach to text matching, where the goal is to jointly select
and align text pieces, such as tokens or sentences, as a justification for the
downstream prediction. Our approach employs optimal transport (OT) to find a
minimal cost alignment between the inputs. However, directly applying OT often
produces dense and therefore uninterpretable alignments. To overcome this
limitation, we introduce novel constrained variants of the OT problem that
result in highly sparse alignments with controllable sparsity. Our model is
end-to-end differentiable using the Sinkhorn algorithm for OT and can be
trained without any alignment annotations. We evaluate our model on the
StackExchange, MultiNews, e-SNLI, and MultiRC datasets. Our model achieves very
sparse rationale selections with high fidelity while preserving prediction
accuracy compared to strong attention baseline models.
