StereoSpike: Depth Learning with a Spiking Neural Network
  Depth estimation is an important computer vision task, useful in particular
for navigation in autonomous vehicles, or for object manipulation in robotics.
Here we solved it using an end-to-end neuromorphic approach, combining two
event-based cameras and a Spiking Neural Network (SNN) with a slightly modified
U-Net-like encoder-decoder architecture, that we named StereoSpike. More
specifically, we used the Multi Vehicle Stereo Event Camera Dataset (MVSEC). It
provides a depth ground-truth, which was used to train StereoSpike in a
supervised manner, using surrogate gradient descent. We propose a novel readout
paradigm to obtain a dense analog prediction -- the depth of each pixel -- from
the spikes of the decoder. We demonstrate that this architecture generalizes
very well, even better than its non-spiking counterparts, leading to
state-of-the-art test accuracy. To the best of our knowledge, it is the first
time that such a large-scale regression problem is solved by a fully spiking
network. Finally, we show that low firing rates (<10%) can be obtained via
regularization, with a minimal cost in accuracy. This means that StereoSpike
could be efficiently implemented on neuromorphic chips, opening the door for
low power and real time embedded systems.
