Meta reinforcement learning as task inference
  Humans achieve efficient learning by relying on prior knowledge about the
structure of naturally occurring tasks. There is considerable interest in
designing reinforcement learning (RL) algorithms with similar properties. This
includes proposals to learn the learning algorithm itself, an idea also known
as meta learning. One formal interpretation of this idea is as a partially
observable multi-task RL problem in which task information is hidden from the
agent. Such unknown task problems can be reduced to Markov decision processes
(MDPs) by augmenting an agent's observations with an estimate of the belief
about the task based on past experience. However estimating the belief state is
intractable in most partially-observed MDPs. We propose a method that
separately learns the policy and the task belief by taking advantage of various
kinds of privileged information. Our approach can be very effective at solving
standard meta-RL environments, as well as a complex continuous control
environment with sparse rewards and requiring long-term memory.
