Prescient teleoperation of humanoid robots
  Humanoid robots could be versatile and intuitive human avatars that operate
remotely in inaccessible places: the robot could reproduce in the remote
location the movements of an operator equipped with a wearable motion capture
device while sending visual feedback to the operator. While substantial
progress has been made on transferring ("retargeting") human motions to
humanoid robots, a major problem preventing the deployment of such systems in
real applications is the presence of communication delays between the human
input and the feedback from the robot: even a few hundred milliseconds of delay
can irreversibly disturb the operator, let alone a few seconds. To overcome
these delays, we introduce a system in which a humanoid robot executes commands
before it actually receives them, so that the visual feedback appears to be
synchronized to the operator, whereas the robot executed the commands in the
past. To do so, the robot continuously predicts future commands by querying a
machine learning model that is trained on past trajectories and conditioned on
the last received commands. In our experiments, an operator was able to
successfully control a humanoid robot (32 degrees of freedom) with stochastic
delays up to 2 seconds in several whole-body manipulation tasks, including
reaching different targets, picking up a bottle, and placing a box at distinct
locations.
