Solving the Real Robot Challenge using Deep Reinforcement Learning
  This paper details our winning submission to Phase 1 of the 2021 Real Robot
Challenge; a challenge in which a three fingered robot must carry a cube along
specified goal trajectories. To solve Phase 1, we use a pure reinforcement
learning approach which requires minimal expert knowledge of the robotic system
or of robotic grasping in general. A sparse, goal-based reward is employed in
conjunction with Hindsight Experience Replay to teach the control policy to
move the cube to the desired x and y coordinates. Simultaneously, a dense
distance-based reward is employed to teach the policy to lift the cube to the
desired z coordinate. The policy is trained in simulation with domain
randomization before being transferred to the real robot for evaluation.
Although performance tends to worsen after this transfer, our best trained
policy can successfully lift the real cube along goal trajectories via the use
of an effective pinching grasp. Our approach outperforms all other submissions,
including those leveraging more traditional robotic control techniques, and is
the first learning-based approach to solve this challenge.
