GCNet: Graph Completion Network for Incomplete Multimodal Learning in
  Conversation
  Conversations have become a critical data format on social media platforms.
Understanding conversation from emotion, content, and other aspects also
attracts increasing attention from researchers due to its widespread
application in human-computer interaction. In real-world environments, we often
encounter the problem of incomplete modalities, which has become a core issue
of conversation understanding. To address this problem, researchers propose
various methods. However, existing approaches are mainly designed for
individual utterances or medical images rather than conversational data, which
cannot exploit temporal and speaker information in conversations. To this end,
we propose a novel framework for incomplete multimodal learning in
conversations, called "Graph Complete Network (GCNet)", filling the gap of
existing works. Our GCNet contains two well-designed graph neural network-based
modules, "Speaker GNN" and "Temporal GNN", to capture temporal and speaker
information in conversations. To make full use of complete and incomplete data
in feature learning, we jointly optimize classification and reconstruction in
an end-to-end manner. To verify the effectiveness of our method, we conduct
experiments on three benchmark conversational datasets. Experimental results
demonstrate that our GCNet is superior to existing state-of-the-art approaches
in incomplete multimodal learning.
