LENS: Localization enhanced by NeRF synthesis
  Neural Radiance Fields (NeRF) have recently demonstrated photo-realistic
results for the task of novel view synthesis. In this paper, we propose to
apply novel view synthesis to the robot relocalization problem: we demonstrate
improvement of camera pose regression thanks to an additional synthetic dataset
rendered by the NeRF class of algorithm. To avoid spawning novel views in
irrelevant places we selected virtual camera locations from NeRF internal
representation of the 3D geometry of the scene. We further improved
localization accuracy of pose regressors using synthesized realistic and
geometry consistent images as data augmentation during training. At the time of
publication, our approach improved state of the art with a 60% lower error on
Cambridge Landmarks and 7-scenes datasets. Hence, the resulting accuracy
becomes comparable to structure-based methods, without any architecture
modification or domain adaptation constraints. Since our method allows almost
infinite generation of training data, we investigated limitations of camera
pose regression depending on size and distribution of data used for training on
public benchmarks. We concluded that pose regression accuracy is mostly bounded
by relatively small and biased datasets rather than capacity of the pose
regression model to solve the localization task.
