ClarQ: A large-scale and diverse dataset for Clarification Question
  Generation
  Question answering and conversational systems are often baffled and need help
clarifying certain ambiguities. However, limitations of existing datasets
hinder the development of large-scale models capable of generating and
utilising clarification questions. In order to overcome these limitations, we
devise a novel bootstrapping framework (based on self-supervision) that assists
in the creation of a diverse, large-scale dataset of clarification questions
based on post-comment tuples extracted from stackexchange. The framework
utilises a neural network based architecture for classifying clarification
questions. It is a two-step method where the first aims to increase the
precision of the classifier and second aims to increase its recall. We
quantitatively demonstrate the utility of the newly created dataset by applying
it to the downstream task of question-answering. The final dataset, ClarQ,
consists of ~2M examples distributed across 173 domains of stackexchange. We
release this dataset in order to foster research into the field of
clarification question generation with the larger goal of enhancing dialog and
question answering systems.
