Sim-To-Real Transfer for Miniature Autonomous Car Racing
  Sim-to-real, a term that describes where a model is trained in a simulator
then transferred to the real world, is a technique that enables faster deep
reinforcement learning (DRL) training. However, differences between the
simulator and the real world often cause the model to perform poorly in the
real world. Domain randomization is a way to bridge the sim-to-real gap by
exposing the model to a wide range of scenarios so that it can generalize to
real-world situations. However, following domain randomization to train an
autonomous car racing model with DRL can lead to undesirable outcomes. Namely,
a model trained with randomization tends to run slower; a higher completion
rate on the testing track comes at the expense of longer lap times. This paper
aims to boost the robustness of a trained race car model without compromising
racing lap times. For a training track and a testing track having the same
shape (and same optimal paths), but with different lighting, background, etc.,
we first train a model (teacher model) that overfits the training track, moving
along a near optimal path. We then use this model to teach a student model the
correct actions along with randomization. With our method, a model with 18.4\%
completion rate on the testing track is able to help teach a student model with
52\% completion. Moreover, over an average of 50 trials, the student is able to
finish a lap 0.23 seconds faster than the teacher. This 0.23 second gap is
significant in tight races, with lap times of about 10 to 12 seconds.
