What Does The User Want? Information Gain for Hierarchical Dialogue
  Policy Optimisation
  The dialogue management component of a task-oriented dialogue system is
typically optimised via reinforcement learning (RL). Optimisation via RL is
highly susceptible to sample inefficiency and instability. The hierarchical
approach called Feudal Dialogue Management takes a step towards more efficient
learning by decomposing the action space. However, it still suffers from
instability due to the reward only being provided at the end of the dialogue.
We propose the usage of an intrinsic reward based on information gain to
address this issue. Our proposed reward favours actions that resolve
uncertainty or query the user whenever necessary. It enables the policy to
learn how to retrieve the users' needs efficiently, which is an integral aspect
in every task-oriented conversation. Our algorithm, which we call FeudalGain,
achieves state-of-the-art results in most environments of the PyDial framework,
outperforming much more complex approaches. We confirm the sample efficiency
and stability of our algorithm through experiments in simulation and a human
trial.
