Reasoning Visual Dialog with Sparse Graph Learning and Knowledge
  Transfer
  Visual dialog is a task of answering a sequence of questions grounded in an
image using the previous dialog history as context. In this paper, we study how
to address two fundamental challenges for this task: (1) reasoning over
underlying semantic structures among dialog rounds and (2) identifying several
appropriate answers to the given question. To address these challenges, we
propose a Sparse Graph Learning (SGL) method to formulate visual dialog as a
graph structure learning task. SGL infers inherently sparse dialog structures
by incorporating binary and score edges and leveraging a new structural loss
function. Next, we introduce a Knowledge Transfer (KT) method that extracts the
answer predictions from the teacher model and uses them as pseudo labels. We
propose KT to remedy the shortcomings of single ground-truth labels, which
severely limit the ability of a model to obtain multiple reasonable answers. As
a result, our proposed model significantly improves reasoning capability
compared to baseline methods and outperforms the state-of-the-art approaches on
the VisDial v1.0 dataset. The source code is available at
https://github.com/gicheonkang/SGLKT-VisDial.
