Transferring Dexterous Manipulation from GPU Simulation to a Remote
  Real-World TriFinger
  We present a system for learning a challenging dexterous manipulation task
involving moving a cube to an arbitrary 6-DoF pose with only 3-fingers trained
with NVIDIA's IsaacGym simulator. We show empirical benefits, both in
simulation and sim-to-real transfer, of using keypoints as opposed to
position+quaternion representations for the object pose in 6-DoF for policy
observations and in reward calculation to train a model-free reinforcement
learning agent. By utilizing domain randomization strategies along with the
keypoint representation of the pose of the manipulated object, we achieve a
high success rate of 83% on a remote TriFinger system maintained by the
organizers of the Real Robot Challenge. With the aim of assisting further
research in learning in-hand manipulation, we make the codebase of our system,
along with trained checkpoints that come with billions of steps of experience
available, at https://s2r2-ig.github.io
