SOLAR: Deep Structured Representations for Model-Based Reinforcement
  Learning
  Model-based reinforcement learning (RL) has proven to be a data efficient
approach for learning control tasks but is difficult to utilize in domains with
complex observations such as images. In this paper, we present a method for
learning representations that are suitable for iterative model-based policy
improvement, even when the underlying dynamical system has complex dynamics and
image observations, in that these representations are optimized for inferring
simple dynamics and cost models given data from the current policy. This
enables a model-based RL method based on the linear-quadratic regulator (LQR)
to be used for systems with image observations. We evaluate our approach on a
range of robotics tasks, including manipulation with a real-world robotic arm
directly from images. We find that our method produces substantially better
final performance than other model-based RL methods while being significantly
more efficient than model-free RL.
