ELSIM: End-to-end learning of reusable skills through intrinsic
  motivation
  Taking inspiration from developmental learning, we present a novel
reinforcement learning architecture which hierarchically learns and represents
self-generated skills in an end-to-end way. With this architecture, an agent
focuses only on task-rewarded skills while keeping the learning process of
skills bottom-up. This bottom-up approach allows to learn skills that 1- are
transferable across tasks, 2- improves exploration when rewards are sparse. To
do so, we combine a previously defined mutual information objective with a
novel curriculum learning algorithm, creating an unlimited and explorable tree
of skills. We test our agent on simple gridworld environments to understand and
visualize how the agent distinguishes between its skills. Then we show that our
approach can scale on more difficult MuJoCo environments in which our agent is
able to build a representation of skills which improve over a baseline both
transfer learning and exploration when rewards are sparse.
