Learning from Real Users: Rating Dialogue Success with Neural Networks
  for Reinforcement Learning in Spoken Dialogue Systems
  To train a statistical spoken dialogue system (SDS) it is essential that an
accurate method for measuring task success is available. To date training has
relied on presenting a task to either simulated or paid users and inferring the
dialogue's success by observing whether this presented task was achieved or
not. Our aim however is to be able to learn from real users acting under their
own volition, in which case it is non-trivial to rate the success as any prior
knowledge of the task is simply unavailable. User feedback may be utilised but
has been found to be inconsistent. Hence, here we present two neural network
models that evaluate a sequence of turn-level features to rate the success of a
dialogue. Importantly these models make no use of any prior knowledge of the
user's task. The models are trained on dialogues generated by a simulated user
and the best model is then used to train a policy on-line which is shown to
perform at least as well as a baseline system using prior knowledge of the
user's task. We note that the models should also be of interest for evaluating
SDS and for monitoring a dialogue in rule-based SDS.
