Unicorn: Continual Learning with a Universal, Off-policy Agent
  Some real-world domains are best characterized as a single task, but for
others this perspective is limiting. Instead, some tasks continually grow in
complexity, in tandem with the agent's competence. In continual learning, also
referred to as lifelong learning, there are no explicit task boundaries or
curricula. As learning agents have become more powerful, continual learning
remains one of the frontiers that has resisted quick progress. To test
continual learning capabilities we consider a challenging 3D domain with an
implicit sequence of tasks and sparse rewards. We propose a novel agent
architecture called Unicorn, which demonstrates strong continual learning and
outperforms several baseline agents on the proposed domain. The agent achieves
this by jointly representing and learning multiple policies efficiently, using
a parallel off-policy learning setup.
