HyperDynamics: Meta-Learning Object and Agent Dynamics with
  Hypernetworks
  We propose HyperDynamics, a dynamics meta-learning framework that conditions
on an agent's interactions with the environment and optionally its visual
observations, and generates the parameters of neural dynamics models based on
inferred properties of the dynamical system. Physical and visual properties of
the environment that are not part of the low-dimensional state yet affect its
temporal dynamics are inferred from the interaction history and visual
observations, and are implicitly captured in the generated parameters. We test
HyperDynamics on a set of object pushing and locomotion tasks. It outperforms
existing dynamics models in the literature that adapt to environment variations
by learning dynamics over high dimensional visual observations, capturing the
interactions of the agent in recurrent state representations, or using
gradient-based meta-optimization. We also show our method matches the
performance of an ensemble of separately trained experts, while also being able
to generalize well to unseen environment variations at test time. We attribute
its good performance to the multiplicative interactions between the inferred
system properties -- captured in the generated parameters -- and the
low-dimensional state representation of the dynamical system.
