Memory Fusion Network for Multi-view Sequential Learning
  Multi-view sequential learning is a fundamental problem in machine learning
dealing with multi-view sequences. In a multi-view sequence, there exists two
forms of interactions between different views: view-specific interactions and
cross-view interactions. In this paper, we present a new neural architecture
for multi-view sequential learning called the Memory Fusion Network (MFN) that
explicitly accounts for both interactions in a neural architecture and
continuously models them through time. The first component of the MFN is called
the System of LSTMs, where view-specific interactions are learned in isolation
through assigning an LSTM function to each view. The cross-view interactions
are then identified using a special attention mechanism called the Delta-memory
Attention Network (DMAN) and summarized through time with a Multi-view Gated
Memory. Through extensive experimentation, MFN is compared to various proposed
approaches for multi-view sequential learning on multiple publicly available
benchmark datasets. MFN outperforms all the existing multi-view approaches.
Furthermore, MFN outperforms all current state-of-the-art models, setting new
state-of-the-art results for these multi-view datasets.
