Dyadformer: A Multi-modal Transformer for Long-Range Modeling of Dyadic
  Interactions
  Personality computing has become an emerging topic in computer vision, due to
the wide range of applications it can be used for. However, most works on the
topic have focused on analyzing the individual, even when applied to
interaction scenarios, and for short periods of time. To address these
limitations, we present the Dyadformer, a novel multi-modal multi-subject
Transformer architecture to model individual and interpersonal features in
dyadic interactions using variable time windows, thus allowing the capture of
long-term interdependencies. Our proposed cross-subject layer allows the
network to explicitly model interactions among subjects through attentional
operations. This proof-of-concept approach shows how multi-modality and joint
modeling of both interactants for longer periods of time helps to predict
individual attributes. With Dyadformer, we improve state-of-the-art
self-reported personality inference results on individual subjects on the UDIVA
v0.5 dataset.
