Latent Skill Planning for Exploration and Transfer
  To quickly solve new tasks in complex environments, intelligent agents need
to build up reusable knowledge. For example, a learned world model captures
knowledge about the environment that applies to new tasks. Similarly, skills
capture general behaviors that can apply to new tasks. In this paper, we
investigate how these two approaches can be integrated into a single
reinforcement learning agent. Specifically, we leverage the idea of partial
amortization for fast adaptation at test time. For this, actions are produced
by a policy that is learned over time while the skills it conditions on are
chosen using online planning. We demonstrate the benefits of our design
decisions across a suite of challenging locomotion tasks and demonstrate
improved sample efficiency in single tasks as well as in transfer from one task
to another, as compared to competitive baselines. Videos are available at:
https://sites.google.com/view/latent-skill-planning/
