Deep Learning for Human Affect Recognition: Insights and New
  Developments
  Automatic human affect recognition is a key step towards more natural
human-computer interaction. Recent trends include recognition in the wild using
a fusion of audiovisual and physiological sensors, a challenging setting for
conventional machine learning algorithms. Since 2010, novel deep learning
algorithms have been applied increasingly in this field. In this paper, we
review the literature on human affect recognition between 2010 and 2017, with a
special focus on approaches using deep neural networks. By classifying a total
of 950 studies according to their usage of shallow or deep architectures, we
are able to show a trend towards deep learning. Reviewing a subset of 233
studies that employ deep neural networks, we comprehensively quantify their
applications in this field. We find that deep learning is used for learning of
(i) spatial feature representations, (ii) temporal feature representations, and
(iii) joint feature representations for multimodal sensor data. Exemplary
state-of-the-art architectures illustrate the progress. Our findings show the
role deep architectures will play in human affect recognition, and can serve as
a reference point for researchers working on related applications.
