Grounding Natural Language Commands to StarCraft II Game States for
  Narration-Guided Reinforcement Learning
  While deep reinforcement learning techniques have led to agents that are
successfully able to learn to perform a number of tasks that had been
previously unlearnable, these techniques are still susceptible to the
longstanding problem of {\em reward sparsity}. This is especially true for
tasks such as training an agent to play StarCraft II, a real-time strategy game
where reward is only given at the end of a game which is usually very long.
While this problem can be addressed through reward shaping, such approaches
typically require a human expert with specialized knowledge. Inspired by the
vision of enabling reward shaping through the more-accessible paradigm of
natural-language narration, we investigate to what extent we can contextualize
these narrations by grounding them to the goal-specific states. We present a
mutual-embedding model using a multi-input deep-neural network that projects a
sequence of natural language commands into the same high-dimensional
representation space as corresponding goal states. We show that using this
model we can learn an embedding space with separable and distinct clusters that
accurately maps natural-language commands to corresponding game states . We
also discuss how this model can allow for the use of narrations as a robust
form of reward shaping to improve RL performance and efficiency.
