What If: Generating Code to Answer Simulation Questions
  Many texts, especially in chemistry and biology, describe complex processes.
We focus on texts that describe a chemical reaction process and questions that
ask about the process's outcome under different environmental conditions. To
answer questions about such processes, one needs to understand the interactions
between the different entities involved in the process and to simulate their
state transitions during the process execution under different conditions. A
state transition is defined as the memory modification the program does to the
variables during the execution. We hypothesize that generating code and
executing it to simulate the process will allow answering such questions. We,
therefore, define a domain-specific language (DSL) to represent processes. We
contribute to the community a unique dataset curated by chemists and annotated
by computer scientists. The dataset is composed of process texts, simulation
questions, and their corresponding computer codes represented by the DSL.We
propose a neural program synthesis approach based on reinforcement learning
with a novel state-transition semantic reward. The novel reward is based on the
run-time semantic similarity between the predicted code and the reference code.
This allows simulating complex process transitions and thus answering
simulation questions. Our approach yields a significant boost in accuracy for
simulation questions: 88\% accuracy as opposed to 83\% accuracy of the
state-of-the-art neural program synthesis approaches and 54\% accuracy of
state-of-the-art end-to-end text-based approaches.
