Towards Robustifying NLI Models Against Lexical Dataset Biases
  While deep learning models are making fast progress on the task of Natural
Language Inference, recent studies have also shown that these models achieve
high accuracy by exploiting several dataset biases, and without deep
understanding of the language semantics. Using contradiction-word bias and
word-overlapping bias as our two bias examples, this paper explores both
data-level and model-level debiasing methods to robustify models against
lexical dataset biases. First, we debias the dataset through data augmentation
and enhancement, but show that the model bias cannot be fully removed via this
method. Next, we also compare two ways of directly debiasing the model without
knowing what the dataset biases are in advance. The first approach aims to
remove the label bias at the embedding level. The second approach employs a
bag-of-words sub-model to capture the features that are likely to exploit the
bias and prevents the original model from learning these biased features by
forcing orthogonality between these two sub-models. We performed evaluations on
new balanced datasets extracted from the original MNLI dataset as well as the
NLI stress tests, and show that the orthogonality approach is better at
debiasing the model while maintaining competitive overall accuracy. Our code
and data are available at: https://github.com/owenzx/LexicalDebias-ACL2020
