Generative Adversarial Imitation Learning for End-to-End Autonomous
  Driving on Urban Environments
  Autonomous driving is a complex task, which has been tackled since the first
self-driving car ALVINN in 1989, with a supervised learning approach, or
behavioral cloning (BC). In BC, a neural network is trained with state-action
pairs that constitute the training set made by an expert, i.e., a human driver.
However, this type of imitation learning does not take into account the
temporal dependencies that might exist between actions taken in different
moments of a navigation trajectory. These type of tasks are better handled by
reinforcement learning (RL) algorithms, which need to define a reward function.
On the other hand, more recent approaches to imitation learning, such as
Generative Adversarial Imitation Learning (GAIL), can train policies without
explicitly requiring to define a reward function, allowing an agent to learn by
trial and error directly on a training set of expert trajectories. In this
work, we propose two variations of GAIL for autonomous navigation of a vehicle
in the realistic CARLA simulation environment for urban scenarios. Both of them
use the same network architecture, which process high dimensional image input
from three frontal cameras, and other nine continuous inputs representing the
velocity, the next point from the sparse trajectory and a high-level driving
command. We show that both of them are capable of imitating the expert
trajectory from start to end after training ends, but the GAIL loss function
that is augmented with BC outperforms the former in terms of convergence time
and training stability.
