e-QRAQ: A Multi-turn Reasoning Dataset and Simulator with Explanations
  In this paper we present a new dataset and user simulator e-QRAQ (explainable
Query, Reason, and Answer Question) which tests an Agent's ability to read an
ambiguous text; ask questions until it can answer a challenge question; and
explain the reasoning behind its questions and answer. The User simulator
provides the Agent with a short, ambiguous story and a challenge question about
the story. The story is ambiguous because some of the entities have been
replaced by variables. At each turn the Agent may ask for the value of a
variable or try to answer the challenge question. In response the User
simulator provides a natural language explanation of why the Agent's query or
answer was useful in narrowing down the set of possible answers, or not. To
demonstrate one potential application of the e-QRAQ dataset, we train a new
neural architecture based on End-to-End Memory Networks to successfully
generate both predictions and partial explanations of its current understanding
of the problem. We observe a strong correlation between the quality of the
prediction and explanation.
