Curiosity-Driven Multi-Criteria Hindsight Experience Replay
  Dealing with sparse rewards is a longstanding challenge in reinforcement
learning. The recent use of hindsight methods have achieved success on a
variety of sparse-reward tasks, but they fail on complex tasks such as stacking
multiple blocks with a robot arm in simulation. Curiosity-driven exploration
using the prediction error of a learned dynamics model as an intrinsic reward
has been shown to be effective for exploring a number of sparse-reward
environments. We present a method that combines hindsight with curiosity-driven
exploration and curriculum learning in order to solve the challenging
sparse-reward block stacking task. We are the first to stack more than two
blocks using only sparse reward without human demonstrations.
