Logic Constrained Pointer Networks for Interpretable Textual Similarity
  Systematically discovering semantic relationships in text is an important and
extensively studied area in Natural Language Processing, with various tasks
such as entailment, semantic similarity, etc. Decomposability of sentence-level
scores via subsequence alignments has been proposed as a way to make models
more interpretable. We study the problem of aligning components of sentences
leading to an interpretable model for semantic textual similarity. In this
paper, we introduce a novel pointer network based model with a sentinel gating
function to align constituent chunks, which are represented using BERT. We
improve this base model with a loss function to equally penalize misalignments
in both sentences, ensuring the alignments are bidirectional. Finally, to guide
the network with structured external knowledge, we introduce first-order logic
constraints based on ConceptNet and syntactic knowledge. The model achieves an
F1 score of 97.73 and 96.32 on the benchmark SemEval datasets for the chunk
alignment task, showing large improvements over the existing solutions. Source
code is available at
https://github.com/manishb89/interpretable_sentence_similarity
