Unsupervised Multimodal Language Representations using Convolutional
  Autoencoders
  Multimodal Language Analysis is a demanding area of research, since it is
associated with two requirements: combining different modalities and capturing
temporal information. During the last years, several works have been proposed
in the area, mostly centered around supervised learning in downstream tasks. In
this paper we propose extracting unsupervised Multimodal Language
representations that are universal and can be applied to different tasks.
Towards this end, we map the word-level aligned multimodal sequences to 2-D
matrices and then use Convolutional Autoencoders to learn embeddings by
combining multiple datasets. Extensive experimentation on Sentiment Analysis
(MOSEI) and Emotion Recognition (IEMOCAP) indicate that the learned
representations can achieve near-state-of-the-art performance with just the use
of a Logistic Regression algorithm for downstream classification. It is also
shown that our method is extremely lightweight and can be easily generalized to
other tasks and unseen data with small performance drop and almost the same
number of parameters. The proposed multimodal representation models are
open-sourced and will help grow the applicability of Multimodal Language.
