Benchmarking Image Retrieval for Visual Localization
  Visual localization, i.e., camera pose estimation in a known scene, is a core
component of technologies such as autonomous driving and augmented reality.
State-of-the-art localization approaches often rely on image retrieval
techniques for one of two tasks: (1) provide an approximate pose estimate or
(2) determine which parts of the scene are potentially visible in a given query
image. It is common practice to use state-of-the-art image retrieval algorithms
for these tasks. These algorithms are often trained for the goal of retrieving
the same landmark under a large range of viewpoint changes. However, robustness
to viewpoint changes is not necessarily desirable in the context of visual
localization. This paper focuses on understanding the role of image retrieval
for multiple visual localization tasks. We introduce a benchmark setup and
compare state-of-the-art retrieval representations on multiple datasets. We
show that retrieval performance on classical landmark retrieval/recognition
tasks correlates only for some but not all tasks to localization performance.
This indicates a need for retrieval approaches specifically designed for
localization tasks. Our benchmark and evaluation protocols are available at
https://github.com/naver/kapture-localization.
