Deep Reinforcement Learning for Navigation in AAA Video Games
  In video games, non-player characters (NPCs) are used to enhance the players'
experience in a variety of ways, e.g., as enemies, allies, or innocent
bystanders. A crucial component of NPCs is navigation, which allows them to
move from one point to another on the map. The most popular approach for NPC
navigation in the video game industry is to use a navigation mesh (NavMesh),
which is a graph representation of the map, with nodes and edges indicating
traversable areas. Unfortunately, complex navigation abilities that extend the
character's capacity for movement, e.g., grappling hooks, jetpacks,
teleportation, or double-jumps, increases the complexity of the NavMesh, making
it intractable in many practical scenarios. Game designers are thus constrained
to only add abilities that can be handled by a NavMesh if they want to have NPC
navigation. As an alternative, we propose to use Deep Reinforcement Learning
(Deep RL) to learn how to navigate 3D maps using any navigation ability. We
test our approach on complex 3D environments in the Unity game engine that are
notably an order of magnitude larger than maps typically used in the Deep RL
literature. One of these maps is directly modeled after a Ubisoft AAA game. We
find that our approach performs surprisingly well, achieving at least $90\%$
success rate on all tested scenarios. A video of our results is available at
https://youtu.be/WFIf9Wwlq8M.
