Improving the Robustness of QA Models to Challenge Sets with Variational
  Question-Answer Pair Generation
  Question answering (QA) models for reading comprehension have achieved
human-level accuracy on in-distribution test sets. However, they have been
demonstrated to lack robustness to challenge sets, whose distribution is
different from that of training sets. Existing data augmentation methods
mitigate this problem by simply augmenting training sets with synthetic
examples sampled from the same distribution as the challenge sets. However,
these methods assume that the distribution of a challenge set is known a
priori, making them less applicable to unseen challenge sets. In this study, we
focus on question-answer pair generation (QAG) to mitigate this problem. While
most existing QAG methods aim to improve the quality of synthetic examples, we
conjecture that diversity-promoting QAG can mitigate the sparsity of training
sets and lead to better robustness. We present a variational QAG model that
generates multiple diverse QA pairs from a paragraph. Our experiments show that
our method can improve the accuracy of 12 challenge sets, as well as the
in-distribution accuracy. Our code and data are available at
https://github.com/KazutoshiShinoda/VQAG.
