Data-Efficient Learning for Complex and Real-Time Physical Problem
  Solving using Augmented Simulation
  Humans quickly solve tasks in novel systems with complex dynamics, without
requiring much interaction. While deep reinforcement learning algorithms have
achieved tremendous success in many complex tasks, these algorithms need a
large number of samples to learn meaningful policies. In this paper, we present
a task for navigating a marble to the center of a circular maze. While this
system is very intuitive and easy for humans to solve, it can be very difficult
and inefficient for standard reinforcement learning algorithms to learn
meaningful policies. We present a model that learns to move a marble in the
complex environment within minutes of interacting with the real system.
Learning consists of initializing a physics engine with parameters estimated
using data from the real system. The error in the physics engine is then
corrected using Gaussian process regression, which is used to model the
residual between real observations and physics engine simulations. The physics
engine augmented with the residual model is then used to control the marble in
the maze environment using a model-predictive feedback over a receding horizon.
To the best of our knowledge, this is the first time that a hybrid model
consisting of a full physics engine along with a statistical function
approximator has been used to control a complex physical system in real-time
using nonlinear model-predictive control (NMPC).
