EgoMap: Projective mapping and structured egocentric memory for Deep RL
  Tasks involving localization, memorization and planning in partially
observable 3D environments are an ongoing challenge in Deep Reinforcement
Learning. We present EgoMap, a spatially structured neural memory architecture.
EgoMap augments a deep reinforcement learning agent's performance in 3D
environments on challenging tasks with multi-step objectives. The EgoMap
architecture incorporates several inductive biases including a differentiable
inverse projection of CNN feature vectors onto a top-down spatially structured
map. The map is updated with ego-motion measurements through a differentiable
affine transform. We show this architecture outperforms both standard recurrent
agents and state of the art agents with structured memory. We demonstrate that
incorporating these inductive biases into an agent's architecture allows for
stable training with reward alone, circumventing the expense of acquiring and
labelling expert trajectories. A detailed ablation study demonstrates the
impact of key aspects of the architecture and through extensive qualitative
analysis, we show how the agent exploits its structured internal memory to
achieve higher performance.
