Intuitive Physics Guided Exploration for Sample Efficient Sim2real
  Transfer
  Physics-based reinforcement learning tasks can benefit from simplified
physics simulators as they potentially allow near-optimal policies to be
learned in simulation. However, such simulators require the latent factors
(e.g. mass, friction coefficient etc.) of the associated objects and other
environment-specific factors (e.g. wind speed, air density etc.) to be
accurately specified, without which, it could take considerable additional
learning effort to adapt the learned simulation policy to the real environment.
As such a complete specification can be impractical, in this paper, we instead,
focus on learning task-specific estimates of latent factors which allow the
approximation of real world trajectories in an ideal simulation environment.
Specifically, we propose two new concepts: a) action grouping - the idea that
certain types of actions are closely associated with the estimation of certain
latent factors, and; b) partial grounding - the idea that simulation of
task-specific dynamics may not need precise estimation of all the latent
factors. We first introduce intuitive action groupings based on human physics
knowledge and experience, which is then used to design novel strategies for
interacting with the real environment. Next, we describe how prior knowledge of
a task in a given environment can be used to extract the relative importance of
different latent factors, and how this can be used to inform partial grounding,
which enables efficient learning of the task in any arbitrary environment. We
demonstrate our approach in a range of physics based tasks, and show that it
achieves superior performance relative to other baselines, using only a limited
number of real-world interactions.
