Curriculum Learning with Hindsight Experience Replay for Sequential
  Object Manipulation Tasks
  Learning complex tasks from scratch is challenging and often impossible for
humans as well as for artificial agents. A curriculum can be used instead,
which decomposes a complex task (target task) into a sequence of source tasks
(the curriculum). Each source task is a simplified version of the next source
task with increasing complexity. Learning then occurs gradually by training on
each source task while using knowledge from the curriculum's prior source
tasks. In this study, we present a new algorithm that combines curriculum
learning with Hindsight Experience Replay (HER), to learn sequential object
manipulation tasks for multiple goals and sparse feedback. The algorithm
exploits the recurrent structure inherent in many object manipulation tasks and
implements the entire learning process in the original simulation without
adjusting it to each source task. We have tested our algorithm on three
challenging throwing tasks and show vast improvements compared to vanilla-HER.
