Deep Recurrent Q-Learning vs Deep Q-Learning on a simple Partially
  Observable Markov Decision Process with Minecraft
  Deep Q-Learning has been successfully applied to a wide variety of tasks in
the past several years. However, the architecture of the vanilla Deep Q-Network
is not suited to deal with partially observable environments such as 3D video
games. For this, recurrent layers have been added to the Deep Q-Network in
order to allow it to handle past dependencies. We here use Minecraft for its
customization advantages and design two very simple missions that can be frames
as Partially Observable Markov Decision Process. We compare on these missions
the Deep Q-Network and the Deep Recurrent Q-Network in order to see if the
latter, which is trickier and longer to train, is always the best architecture
when the agent has to deal with partial observability.
