Visual Dialogue State Tracking for Question Generation
  GuessWhat?! is a visual dialogue task between a guesser and an oracle. The
guesser aims to locate an object supposed by the oracle oneself in an image by
asking a sequence of Yes/No questions. Asking proper questions with the
progress of dialogue is vital for achieving successful final guess. As a
result, the progress of dialogue should be properly represented and tracked.
Previous models for question generation pay less attention on the
representation and tracking of dialogue states, and therefore are prone to
asking low quality questions such as repeated questions. This paper proposes
visual dialogue state tracking (VDST) based method for question generation. A
visual dialogue state is defined as the distribution on objects in the image as
well as representations of objects. Representations of objects are updated with
the change of the distribution on objects. An object-difference based attention
is used to decode new question. The distribution on objects is updated by
comparing the question-answer pair and objects. Experimental results on
GuessWhat?! dataset show that our model significantly outperforms existing
methods and achieves new state-of-the-art performance. It is also noticeable
that our model reduces the rate of repeated questions from more than 50% to
21.9% compared with previous state-of-the-art methods.
