A User Simulator for Task-Completion Dialogues
  Despite widespread interests in reinforcement-learning for task-oriented
dialogue systems, several obstacles can frustrate research and development
progress. First, reinforcement learners typically require interaction with the
environment, so conventional dialogue corpora cannot be used directly. Second,
each task presents specific challenges, requiring separate corpus of
task-specific annotated data. Third, collecting and annotating human-machine or
human-human conversations for task-oriented dialogues requires extensive domain
knowledge. Because building an appropriate dataset can be both financially
costly and time-consuming, one popular approach is to build a user simulator
based upon a corpus of example dialogues. Then, one can train reinforcement
learning agents in an online fashion as they interact with the simulator.
Dialogue agents trained on these simulators can serve as an effective starting
point. Once agents master the simulator, they may be deployed in a real
environment to interact with humans, and continue to be trained online. To ease
empirical algorithmic comparisons in dialogues, this paper introduces a new,
publicly available simulation framework, where our simulator, designed for the
movie-booking domain, leverages both rules and collected data. The simulator
supports two tasks: movie ticket booking and movie seeking. Finally, we
demonstrate several agents and detail the procedure to add and test your own
agent in the proposed framework.
