Adaptive Synthetic Characters for Military Training
  Behaviors of the synthetic characters in current military simulations are
limited since they are generally generated by rule-based and reactive
computational models with minimal intelligence. Such computational models
cannot adapt to reflect the experience of the characters, resulting in brittle
intelligence for even the most effective behavior models devised via costly and
labor-intensive processes. Observation-based behavior model adaptation that
leverages machine learning and the experience of synthetic entities in
combination with appropriate prior knowledge can address the issues in the
existing computational behavior models to create a better training experience
in military training simulations. In this paper, we introduce a framework that
aims to create autonomous synthetic characters that can perform coherent
sequences of believable behavior while being aware of human trainees and their
needs within a training simulation. This framework brings together three
mutually complementary components. The first component is a Unity-based
simulation environment - Rapid Integration and Development Environment (RIDE) -
supporting One World Terrain (OWT) models and capable of running and supporting
machine learning experiments. The second is Shiva, a novel multi-agent
reinforcement and imitation learning framework that can interface with a
variety of simulation environments, and that can additionally utilize a variety
of learning algorithms. The final component is the Sigma Cognitive Architecture
that will augment the behavior models with symbolic and probabilistic reasoning
capabilities. We have successfully created proof-of-concept behavior models
leveraging this framework on realistic terrain as an essential step towards
bringing machine learning into military simulations.
