Learning to Drive Off Road on Smooth Terrain in Unstructured
  Environments Using an On-Board Camera and Sparse Aerial Images
  We present a method for learning to drive on smooth terrain while
simultaneously avoiding collisions in challenging off-road and unstructured
outdoor environments using only visual inputs. Our approach applies a hybrid
model-based and model-free reinforcement learning method that is entirely
self-supervised in labeling terrain roughness and collisions using on-board
sensors. Notably, we provide both first-person and overhead aerial image inputs
to our model. We find that the fusion of these complementary inputs improves
planning foresight and makes the model robust to visual obstructions. Our
results show the ability to generalize to environments with plentiful
vegetation, various types of rock, and sandy trails. During evaluation, our
policy attained 90% smooth terrain traversal and reduced the proportion of
rough terrain driven over by 6.1 times compared to a model using only
first-person imagery.
