Multimodal analysis of the predictability of hand-gesture properties
  Embodied conversational agents benefit from being able to accompany their
speech with gestures. Although many data-driven approaches to gesture
generation have been proposed in recent years, it is still unclear whether such
systems can consistently generate gestures that convey meaning. We investigate
which gesture properties (phase, category, and semantics) can be predicted from
speech text and/or audio using contemporary deep learning. In extensive
experiments, we show that gesture properties related to gesture meaning
(semantics and category) are predictable from text features (time-aligned
FastText embeddings) alone, but not from prosodic audio features, while
rhythm-related gesture properties (phase) on the other hand can be predicted
from audio features better than from text. These results are encouraging as
they indicate that it is possible to equip an embodied agent with content-wise
meaningful co-speech gestures using a machine-learning model.
