Predicting Sim-to-Real Transfer with Probabilistic Dynamics Models
  We propose a method to predict the sim-to-real transfer performance of RL
policies. Our transfer metric simplifies the selection of training setups (such
as algorithm, hyperparameters, randomizations) and policies in simulation,
without the need for extensive and time-consuming real-world rollouts. A
probabilistic dynamics model is trained alongside the policy and evaluated on a
fixed set of real-world trajectories to obtain the transfer metric. Experiments
show that the transfer metric is highly correlated with policy performance in
both simulated and real-world robotic environments for complex manipulation
tasks. We further show that the transfer metric can predict the effect of
training setups on policy transfer performance.
