Fully Autonomous Real-World Reinforcement Learning with Applications to
  Mobile Manipulation
  We study how robots can autonomously learn skills that require a combination
of navigation and grasping. While reinforcement learning in principle provides
for automated robotic skill learning, in practice reinforcement learning in the
real world is challenging and often requires extensive instrumentation and
supervision. Our aim is to devise a robotic reinforcement learning system for
learning navigation and manipulation together, in an autonomous way without
human intervention, enabling continual learning under realistic assumptions.
Our proposed system, ReLMM, can learn continuously on a real-world platform
without any environment instrumentation, without human intervention, and
without access to privileged information, such as maps, objects positions, or a
global view of the environment. Our method employs a modularized policy with
components for manipulation and navigation, where manipulation policy
uncertainty drives exploration for the navigation controller, and the
manipulation module provides rewards for navigation. We evaluate our method on
a room cleanup task, where the robot must navigate to and pick up items
scattered on the floor. After a grasp curriculum training phase, ReLMM can
learn navigation and grasping together fully automatically, in around 40 hours
of autonomous real-world training.
