A Multi-Task Learning & Generation Framework: Valence-Arousal, Action
  Units & Primary Expressions
  Over the past few years many research efforts have been devoted to the field
of affect analysis. Various approaches have been proposed for: i) discrete
emotion recognition in terms of the primary facial expressions; ii) emotion
analysis in terms of facial Action Units (AUs), assuming a fixed expression
intensity; iii) dimensional emotion analysis, in terms of valence and arousal
(VA). These approaches can only be effective, if they are developed using
large, appropriately annotated databases, showing behaviors of people
in-the-wild, i.e., in uncontrolled environments. Aff-Wild has been the first,
large-scale, in-the-wild database (including around 1,200,000 frames of 300
videos), annotated in terms of VA. In the vast majority of existing emotion
databases, their annotation is limited to either primary expressions, or
valence-arousal, or action units. In this paper, we first annotate a part
(around $234,000$ frames) of the Aff-Wild database in terms of $8$ AUs and
another part (around $288,000$ frames) in terms of the $7$ basic emotion
categories, so that parts of this database are annotated in terms of VA, as
well as AUs, or primary expressions. Then, we set up and tackle multi-task
learning for emotion recognition, as well as for facial image generation.
Multi-task learning is performed using: i) a deep neural network with shared
hidden layers, which learns emotional attributes by exploiting their
inter-dependencies; ii) a discriminator of a generative adversarial network
(GAN). On the other hand, image generation is implemented through the generator
of the GAN. For these two tasks, we carefully design loss functions that fit
the examined set-up. Experiments are presented which illustrate the good
performance of the proposed approach when applied to the new annotated parts of
the Aff-Wild database.
