Affect-Aware Deep Belief Network Representations for Multimodal
  Unsupervised Deception Detection
  Automated systems that detect the social behavior of deception can enhance
human well-being across medical, social work, and legal domains. Labeled
datasets to train supervised deception detection models can rarely be collected
for real-world, high-stakes contexts. To address this challenge, we propose the
first unsupervised approach for detecting real-world, high-stakes deception in
videos without requiring labels. This paper presents our novel approach for
affect-aware unsupervised Deep Belief Networks (DBN) to learn discriminative
representations of deceptive and truthful behavior. Drawing on psychology
theories that link affect and deception, we experimented with unimodal and
multimodal DBN-based approaches trained on facial valence, facial arousal,
audio, and visual features. In addition to using facial affect as a feature on
which DBN models are trained, we also introduce a DBN training procedure that
uses facial affect as an aligner of audio-visual representations. We conducted
classification experiments with unsupervised Gaussian Mixture Model clustering
to evaluate our approaches. Our best unsupervised approach (trained on facial
valence and visual features) achieved an AUC of 80%, outperforming human
ability and performing comparably to fully-supervised models. Our results
motivate future work on unsupervised, affect-aware computational approaches for
detecting deception and other social behaviors in the wild.
