Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based
  Sentiment Analysis
  Aspect-based sentiment analysis (ABSA) aims to predict the sentiment towards
a specific aspect in the text. However, existing ABSA test sets cannot be used
to probe whether a model can distinguish the sentiment of the target aspect
from the non-target aspects. To solve this problem, we develop a simple but
effective approach to enrich ABSA test sets. Specifically, we generate new
examples to disentangle the confounding sentiments of the non-target aspects
from the target aspect's sentiment. Based on the SemEval 2014 dataset, we
construct the Aspect Robustness Test Set (ARTS) as a comprehensive probe of the
aspect robustness of ABSA models. Over 92% data of ARTS show high fluency and
desired sentiment on all aspects by human evaluation. Using ARTS, we analyze
the robustness of nine ABSA models, and observe, surprisingly, that their
accuracy drops by up to 69.73%. We explore several ways to improve aspect
robustness, and find that adversarial training can improve models' performance
on ARTS by up to 32.85%. Our code and new test set are available at
https://github.com/zhijing-jin/ARTS_TestSet
