Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement
  Learning
  Autonomous car racing is a major challenge in robotics. It raises fundamental
problems for classical approaches such as planning minimum-time trajectories
under uncertain dynamics and controlling the car at the limits of its handling.
Besides, the requirement of minimizing the lap time, which is a sparse
objective, and the difficulty of collecting training data from human experts
have also hindered researchers from directly applying learning-based approaches
to solve the problem. In the present work, we propose a learning-based system
for autonomous car racing by leveraging a high-fidelity physical car
simulation, a course-progress proxy reward, and deep reinforcement learning. We
deploy our system in Gran Turismo Sport, a world-leading car simulator known
for its realistic physics simulation of different race cars and tracks, which
is even used to recruit human race car drivers. Our trained policy achieves
autonomous racing performance that goes beyond what had been achieved so far by
the built-in AI, and, at the same time, outperforms the fastest driver in a
dataset of over 50,000 human players.
