Optimizing Trajectories for Highway Driving with Offline Reinforcement
  Learning
  Implementing an autonomous vehicle that is able to output feasible, smooth
and efficient trajectories is a long-standing challenge. Several approaches
have been considered, roughly falling under two categories: rule-based and
learning-based approaches. The rule-based approaches, while guaranteeing safety
and feasibility, fall short when it comes to long-term planning and
generalization. The learning-based approaches are able to account for long-term
planning and generalization to unseen situations, but may fail to achieve
smoothness, safety and the feasibility which rule-based approaches ensure.
Hence, combining the two approaches is an evident step towards yielding the
best compromise out of both. We propose a Reinforcement Learning-based
approach, which learns target trajectory parameters for fully autonomous
driving on highways. The trained agent outputs continuous trajectory parameters
based on which a feasible polynomial-based trajectory is generated and
executed. We compare the performance of our agent against four other highway
driving agents. The experiments are conducted in the Sumo simulator, taking
into consideration various realistic, dynamically changing highway scenarios,
including surrounding vehicles with different driver behaviors. We demonstrate
that our offline trained agent, with randomly collected data, learns to drive
smoothly, achieving velocities as close as possible to the desired velocity,
while outperforming the other agents. Code, training data and details available
at: https://nrgit.informatik.uni-freiburg. de/branka.mirchevska/offline-rl-tp.
