Distilling a Hierarchical Policy for Planning and Control via
  Representation and Reinforcement Learning
  We present a hierarchical planning and control framework that enables an
agent to perform various tasks and adapt to a new task flexibly. Rather than
learning an individual policy for each particular task, the proposed framework,
DISH, distills a hierarchical policy from a set of tasks by representation and
reinforcement learning. The framework is based on the idea of latent variable
models that represent high-dimensional observations using low-dimensional
latent variables. The resulting policy consists of two levels of hierarchy: (i)
a planning module that reasons a sequence of latent intentions that would lead
to an optimistic future and (ii) a feedback control policy, shared across the
tasks, that executes the inferred intention. Because the planning is performed
in low-dimensional latent space, the learned policy can immediately be used to
solve or adapt to new tasks without additional training. We demonstrate the
proposed framework can learn compact representations (3- and 1-dimensional
latent states and commands for a humanoid with 197- and 36-dimensional state
features and actions) while solving a small number of imitation tasks, and the
resulting policy is directly applicable to other types of tasks, i.e.,
navigation in cluttered environments. Video: https://youtu.be/HQsQysUWOhg
