Rationales for Sequential Predictions
  Sequence models are a critical component of modern NLP systems, but their
predictions are difficult to explain. We consider model explanations though
rationales, subsets of context that can explain individual model predictions.
We find sequential rationales by solving a combinatorial optimization: the best
rationale is the smallest subset of input tokens that would predict the same
output as the full sequence. Enumerating all subsets is intractable, so we
propose an efficient greedy algorithm to approximate this objective. The
algorithm, which is called greedy rationalization, applies to any model. For
this approach to be effective, the model should form compatible conditional
distributions when making predictions on incomplete subsets of the context.
This condition can be enforced with a short fine-tuning step. We study greedy
rationalization on language modeling and machine translation. Compared to
existing baselines, greedy rationalization is best at optimizing the
combinatorial objective and provides the most faithful rationales. On a new
dataset of annotated sequential rationales, greedy rationales are most similar
to human rationales.
