Data-Driven Reinforcement Learning for Virtual Character Animation
  Control
  Virtual character animation control is a problem for which Reinforcement
Learning (RL) is a viable approach. While current work have applied RL
effectively to portray physics-based skills, social behaviours are challenging
to design reward functions for, due to their lack of physical interaction with
the world. On the other hand, data-driven implementations for these skills have
been limited to supervised learning methods which require extensive training
data and carry constraints on generalisability. In this paper, we propose
RLAnimate, a novel data-driven deep RL approach to address this challenge,
where we combine the strengths of RL together with an ability to learn from a
motion dataset when creating agents. We formalise a mathematical structure for
training agents by refining the conceptual roles of elements such as agents,
environments, states and actions, in a way that leverages attributes of the
character animation domain and model-based RL. An agent trained using our
approach learns versatile animation dynamics to portray multiple behaviours,
using an iterative RL training process, which becomes aware of valid behaviours
via representations learnt from motion capture clips. We demonstrate, by
training agents that portray realistic pointing and waving behaviours, that our
approach requires a significantly lower training time, and substantially fewer
sample episodes to be generated during training relative to state-of-the-art
physics-based RL methods. Also, compared to existing supervised learning-based
animation agents, RLAnimate needs a limited dataset of motion clips to generate
representations of valid behaviours during training.
