Learning Knowledge Graph-based World Models of Textual Environments
  World models improve a learning agent's ability to efficiently operate in
interactive and situated environments. This work focuses on the task of
building world models of text-based game environments. Text-based games, or
interactive narratives, are reinforcement learning environments in which agents
perceive and interact with the world using textual natural language. These
environments contain long, multi-step puzzles or quests woven through a world
that is filled with hundreds of characters, locations, and objects. Our world
model learns to simultaneously: (1) predict changes in the world caused by an
agent's actions when representing the world as a knowledge graph; and (2)
generate the set of contextually relevant natural language actions required to
operate in the world. We frame this task as a Set of Sequences generation
problem by exploiting the inherent structure of knowledge graphs and actions
and introduce both a transformer-based multi-task architecture and a loss
function to train it. A zero-shot ablation study on never-before-seen textual
worlds shows that our methodology significantly outperforms existing textual
world modeling techniques as well as the importance of each of our
contributions.
