Attention-based Curiosity-driven Exploration in Deep Reinforcement
  Learning
  Reinforcement Learning enables to train an agent via interaction with the
environment. However, in the majority of real-world scenarios, the extrinsic
feedback is sparse or not sufficient, thus intrinsic reward formulations are
needed to successfully train the agent. This work investigates and extends the
paradigm of curiosity-driven exploration. First, a probabilistic approach is
taken to exploit the advantages of the attention mechanism, which is
successfully applied in other domains of Deep Learning. Combining them, we
propose new methods, such as AttA2C, an extension of the Actor-Critic
framework. Second, another curiosity-based approach - ICM - is extended. The
proposed model utilizes attention to emphasize features for the dynamic models
within ICM, moreover, we also modify the loss function, resulting in a new
curiosity formulation, which we call rational curiosity. The corresponding
implementation can be found at https://github.com/rpatrik96/AttA2C/.
