Towards Interactive Training of Non-Player Characters in Video Games
  There is a high demand for high-quality Non-Player Characters (NPCs) in video
games. Hand-crafting their behavior is a labor intensive and error prone
engineering process with limited controls exposed to the game designers. We
propose to create such NPC behaviors interactively by training an agent in the
target environment using imitation learning with a human in the loop. While
traditional behavior cloning may fall short of achieving the desired
performance, we show that interactivity can substantially improve it with a
modest amount of human efforts. The model we train is a multi-resolution
ensemble of Markov models, which can be used as is or can be further
"compressed" into a more compact model for inference on consumer devices. We
illustrate our approach on an example in OpenAI Gym, where a human can help to
quickly train an agent with only a handful of interactive demonstrations. We
also outline our experiments with NPC training for a first-person shooter game
currently in development.
