Synthesis of Compositional Animations from Textual Descriptions
  "How can we animate 3D-characters from a movie script or move robots by
simply telling them what we would like them to do?" "How unstructured and
complex can we make a sentence and still generate plausible movements from it?"
These are questions that need to be answered in the long-run, as the field is
still in its infancy. Inspired by these problems, we present a new technique
for generating compositional actions, which handles complex input sentences.
Our output is a 3D pose sequence depicting the actions in the input sentence.
We propose a hierarchical two-stream sequential model to explore a finer
joint-level mapping between natural language sentences and 3D pose sequences
corresponding to the given motion. We learn two manifold representations of the
motion -- one each for the upper body and the lower body movements. Our model
can generate plausible pose sequences for short sentences describing single
actions as well as long compositional sentences describing multiple sequential
and superimposed actions. We evaluate our proposed model on the publicly
available KIT Motion-Language Dataset containing 3D pose data with
human-annotated sentences. Experimental results show that our model advances
the state-of-the-art on text-based motion synthesis in objective evaluations by
a margin of 50%. Qualitative evaluations based on a user study indicate that
our synthesized motions are perceived to be the closest to the ground-truth
motion captures for both short and compositional sentences.
