Multimodal Transformer with Pointer Network for the DSTC8 AVSD Challenge
  Audio-Visual Scene-Aware Dialog (AVSD) is an extension from Video Question
Answering (QA) whereby the dialogue agent is required to generate natural
language responses to address user queries and carry on conversations. This is
a challenging task as it consists of video features of multiple modalities,
including text, visual, and audio features. The agent also needs to learn
semantic dependencies among user utterances and system responses to make
coherent conversations with humans. In this work, we describe our submission to
the AVSD track of the 8th Dialogue System Technology Challenge. We adopt
dot-product attention to combine text and non-text features of input video. We
further enhance the generation capability of the dialogue agent by adopting
pointer networks to point to tokens from multiple source sequences in each
generation step. Our systems achieve high performance in automatic metrics and
obtain 5th and 6th place in human evaluation among all submissions.
