Reinforcement learning for non-prehensile manipulation: Transfer from
  simulation to physical system
  Reinforcement learning has emerged as a promising methodology for training
robot controllers. However, most results have been limited to simulation due to
the need for a large number of samples and the lack of automated-yet-safe data
collection methods. Model-based reinforcement learning methods provide an
avenue to circumvent these challenges, but the traditional concern has been the
mismatch between the simulator and the real world. Here, we show that control
policies learned in simulation can successfully transfer to a physical system,
composed of three Phantom robots pushing an object to various desired target
positions. We use a modified form of the natural policy gradient algorithm for
learning, applied to a carefully identified simulation model. The resulting
policies, trained entirely in simulation, work well on the physical system
without additional training. In addition, we show that training with an
ensemble of models makes the learned policies more robust to modeling errors,
thus compensating for difficulties in system identification.
