Deep Reinforcement Learning for Industrial Insertion Tasks with Visual
  Inputs and Natural Rewards
  Connector insertion and many other tasks commonly found in modern
manufacturing settings involve complex contact dynamics and friction. Since it
is difficult to capture related physical effects with first-order modeling,
traditional control methods often result in brittle and inaccurate controllers,
which have to be manually tuned. Reinforcement learning (RL) methods have been
demonstrated to be capable of learning controllers in such environments from
autonomous interaction with the environment, but running RL algorithms in the
real world poses sample efficiency and safety challenges. Moreover, in
practical real-world settings we cannot assume access to perfect state
information or dense reward signals. In this paper, we consider a variety of
difficult industrial insertion tasks with visual inputs and different natural
reward specifications, namely sparse rewards and goal images. We show that
methods that combine RL with prior information, such as classical controllers
or demonstrations, can solve these tasks from a reasonable amount of real-world
interaction.
