Learning to Win by Reading Manuals in a Monte-Carlo Framework
  Domain knowledge is crucial for effective performance in autonomous control
systems. Typically, human effort is required to encode this knowledge into a
control algorithm. In this paper, we present an approach to language grounding
which automatically interprets text in the context of a complex control
application, such as a game, and uses domain knowledge extracted from the text
to improve control performance. Both text analysis and control strategies are
learned jointly using only a feedback signal inherent to the application. To
effectively leverage textual information, our method automatically extracts the
text segment most relevant to the current game state, and labels it with a
task-centric predicate structure. This labeled text is then used to bias an
action selection policy for the game, guiding it towards promising regions of
the action space. We encode our model for text analysis and game playing in a
multi-layer neural network, representing linguistic decisions via latent
variables in the hidden layers, and game action quality via the output layer.
Operating within the Monte-Carlo Search framework, we estimate model parameters
using feedback from simulated games. We apply our approach to the complex
strategy game Civilization II using the official game manual as the text guide.
Our results show that a linguistically-informed game-playing agent
significantly outperforms its language-unaware counterpart, yielding a 34%
absolute improvement and winning over 65% of games when playing against the
built-in AI of Civilization.
