Curious Exploration and Return-based Memory Restoration for Deep
  Reinforcement Learning
  Reward engineering and designing an incentive reward function are non-trivial
tasks to train agents in complex environments. Furthermore, an inaccurate
reward function may lead to a biased behaviour which is far from an efficient
and optimised behaviour. In this paper, we focus on training a single agent to
score goals with binary success/failure reward function in Half Field Offense
domain. As the major advantage of this research, the agent has no presumption
about the environment which means it only follows the original formulation of
reinforcement learning agents. The main challenge of using such a reward
function is the high sparsity of positive reward signals. To address this
problem, we use a simple prediction-based exploration strategy (called Curious
Exploration) along with a Return-based Memory Restoration (RMR) technique which
tends to remember more valuable memories. The proposed method can be utilized
to train agents in environments with fairly complex state and action spaces.
Our experimental results show that many recent solutions including our baseline
method fail to learn and perform in complex soccer domain. However, the
proposed method can converge easily to the nearly optimal behaviour. The video
presenting the performance of our trained agent is available at
http://bit.ly/HFO_Binary_Reward.
