Learning Navigation by Visual Localization and Trajectory Prediction
  When driving, people make decisions based on current traffic as well as their
desired route. They have a mental map of known routes and are often able to
navigate without needing directions. Current self-driving models improve their
performances when using additional GPS information. Here we aim to push forward
self-driving research and perform route planning even in the absence of GPS.
Our system learns to predict in real-time vehicle's current location and future
trajectory, as a function of time, on a known map, given only the raw video
stream and the intended destination. The GPS signal is available only at
training time, with training data annotation being fully automatic. Different
from other published models, we predict the vehicle's trajectory for up to
seven seconds ahead, from which complete steering, speed and acceleration
information can be derived for the entire time span. Trajectories capture
navigational information on multiple levels, from instant steering commands
that depend on present traffic and obstacles ahead, to longer-term navigation
decisions, towards a specific destination. We collect our dataset with a
regular car and a smartphone that records video and GPS streams. The GPS data
is used to derive ground-truth supervision labels and create an analytical
representation of the traversed map. In tests, our system outperforms published
methods on visual localization and steering and gives accurate navigation
assistance between any two known locations.
