ZippyPoint: Fast Interest Point Detection, Description, and Matching
  through Mixed Precision Discretization
  The design of more complex and powerful neural network models has
significantly advanced the state-of-the-art in local feature detection and
description. These advances can be attributed to deeper networks, improved
training methodologies through self-supervision, or the introduction of new
building blocks, such as graph neural networks for feature matching. However,
in the pursuit of increased performance, efficient architectures that generate
lightweight descriptors have received surprisingly little attention. In this
paper, we investigate the adaptations neural networks for detection and
description require in order to enable their use in embedded platforms. To that
end, we investigate and adapt network quantization techniques for use in
real-time applications. In addition, we revisit common practices in descriptor
quantization and propose the use of a binary descriptor normalization layer,
enabling the generation of distinctive length-invariant binary descriptors.
ZippyPoint, our efficient network, runs at 47.2 fps on the Apple M1 CPU. This
is up to 5x faster than other learned detection and description models, making
it the only real-time learned network. ZippyPoint consistently outperforms all
other binary detection and descriptor methods in visual localization and
homography estimation tasks. Code and trained models will be released upon
publication.
