Algorithmic Improvements for Deep Reinforcement Learning applied to
  Interactive Fiction
  Text-based games are a natural challenge domain for deep reinforcement
learning algorithms. Their state and action spaces are combinatorially large,
their reward function is sparse, and they are partially observable: the agent
is informed of the consequences of its actions through textual feedback. In
this paper we emphasize this latter point and consider the design of a deep
reinforcement learning agent that can play from feedback alone. Our design
recognizes and takes advantage of the structural characteristics of text-based
games. We first propose a contextualisation mechanism, based on accumulated
reward, which simplifies the learning problem and mitigates partial
observability. We then study different methods that rely on the notion that
most actions are ineffectual in any given situation, following Zahavy et al.'s
idea of an admissible action. We evaluate these techniques in a series of
text-based games of increasing difficulty based on the TextWorld framework, as
well as the iconic game Zork. Empirically, we find that these techniques
improve the performance of a baseline deep reinforcement learning agent applied
to text-based games.
