Learning to Scaffold the Development of Robotic Manipulation Skills
  Learning contact-rich, robotic manipulation skills is a challenging problem
due to the high-dimensionality of the state and action space as well as
uncertainty from noisy sensors and inaccurate motor control. To combat these
factors and achieve more robust manipulation, humans actively exploit contact
constraints in the environment. By adopting a similar strategy, robots can also
achieve more robust manipulation. In this paper, we enable a robot to
autonomously modify its environment and thereby discover how to ease
manipulation skill learning. Specifically, we provide the robot with fixtures
that it can freely place within the environment. These fixtures provide hard
constraints that limit the outcome of robot actions. Thereby, they funnel
uncertainty from perception and motor control and scaffold manipulation skill
learning. We propose a learning system that consists of two learning loops. In
the outer loop, the robot positions the fixture in the workspace. In the inner
loop, the robot learns a manipulation skill and after a fixed number of
episodes, returns the reward to the outer loop. Thereby, the robot is
incentivised to place the fixture such that the inner loop quickly achieves a
high reward. We demonstrate our framework both in simulation and in the real
world on three tasks: peg insertion, wrench manipulation and shallow-depth
insertion. We show that manipulation skill learning is dramatically sped up
through this way of scaffolding.
