Learning Complex Dexterous Manipulation with Deep Reinforcement Learning
  and Demonstrations
  Dexterous multi-fingered hands are extremely versatile and provide a generic
way to perform a multitude of tasks in human-centric environments. However,
effectively controlling them remains challenging due to their high
dimensionality and large number of potential contacts. Deep reinforcement
learning (DRL) provides a model-agnostic approach to control complex dynamical
systems, but has not been shown to scale to high-dimensional dexterous
manipulation. Furthermore, deployment of DRL on physical systems remains
challenging due to sample inefficiency. Consequently, the success of DRL in
robotics has thus far been limited to simpler manipulators and tasks. In this
work, we show that model-free DRL can effectively scale up to complex
manipulation tasks with a high-dimensional 24-DoF hand, and solve them from
scratch in simulated experiments. Furthermore, with the use of a small number
of human demonstrations, the sample complexity can be significantly reduced,
which enables learning with sample sizes equivalent to a few hours of robot
experience. The use of demonstrations result in policies that exhibit very
natural movements and, surprisingly, are also substantially more robust.
