On-line Active Reward Learning for Policy Optimisation in Spoken
  Dialogue Systems
  The ability to compute an accurate reward function is essential for
optimising a dialogue policy via reinforcement learning. In real-world
applications, using explicit user feedback as the reward signal is often
unreliable and costly to collect. This problem can be mitigated if the user's
intent is known in advance or data is available to pre-train a task success
predictor off-line. In practice neither of these apply for most real world
applications. Here we propose an on-line learning framework whereby the
dialogue policy is jointly trained alongside the reward model via active
learning with a Gaussian process model. This Gaussian process operates on a
continuous space dialogue representation generated in an unsupervised fashion
using a recurrent neural network encoder-decoder. The experimental results
demonstrate that the proposed framework is able to significantly reduce data
annotation costs and mitigate noisy user feedback in dialogue policy learning.
