Document-editing Assistants and Model-based Reinforcement Learning as a
  Path to Conversational AI
  Intelligent assistants that follow commands or answer simple questions, such
as Siri and Google search, are among the most economically important
applications of AI. Future conversational AI assistants promise even greater
capabilities and a better user experience through a deeper understanding of the
domain, the user, or the user's purposes. But what domain and what methods are
best suited to researching and realizing this promise? In this article we argue
for the domain of voice document editing and for the methods of model-based
reinforcement learning. The primary advantages of voice document editing are
that the domain is tightly scoped and that it provides something for the
conversation to be about (the document) that is delimited and fully accessible
to the intelligent assistant. The advantages of reinforcement learning in
general are that its methods are designed to learn from interaction without
explicit instruction and that it formalizes the purposes of the assistant.
Model-based reinforcement learning is needed in order to genuinely understand
the domain of discourse and thereby work efficiently with the user to achieve
their goals. Together, voice document editing and model-based reinforcement
learning comprise a promising research direction for achieving conversational
AI.
