Action-Affect Classification and Morphing using Multi-Task
  Representation Learning
  Most recent work focused on affect from facial expressions, and not as much
on body. This work focuses on body affect analysis. Affect does not occur in
isolation. Humans usually couple affect with an action in natural interactions;
for example, a person could be talking and smiling. Recognizing body affect in
sequences requires efficient algorithms to capture both the micro movements
that differentiate between happy and sad and the macro variations between
different actions. We depart from traditional approaches for time-series data
analytics by proposing a multi-task learning model that learns a shared
representation that is well-suited for action-affect classification as well as
generation. For this paper we choose Conditional Restricted Boltzmann Machines
to be our building block. We propose a new model that enhances the CRBM model
with a factored multi-task component to become Multi-Task Conditional
Restricted Boltzmann Machines (MTCRBMs). We evaluate our approach on two
publicly available datasets, the Body Affect dataset and the Tower Game
dataset, and show superior classification performance improvement over the
state-of-the-art, as well as the generative abilities of our model.
