Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural
  Networks
  The field of neuromorphic computing promises extremely low-power and
low-latency sensing and processing. Challenges in transferring learning
algorithms from traditional artificial neural networks (ANNs) to spiking neural
networks (SNNs) have so far prevented their application to large-scale, complex
regression tasks. Furthermore, realizing a truly asynchronous and fully
neuromorphic pipeline that maximally attains the abovementioned benefits
involves rethinking the way in which this pipeline takes in and accumulates
information. In the case of perception, spikes would be passed as-is and
one-by-one between an event camera and an SNN, meaning all temporal integration
of information must happen inside the network. In this article, we tackle these
two problems. We focus on the complex task of learning to estimate optical flow
from event-based camera inputs in a self-supervised manner, and modify the
state-of-the-art ANN training pipeline to encode minimal temporal information
in its inputs. Moreover, we reformulate the self-supervised loss function for
event-based optical flow to improve its convexity. We perform experiments with
various types of recurrent ANNs and SNNs using the proposed pipeline.
Concerning SNNs, we investigate the effects of elements such as parameter
initialization and optimization, surrogate gradient shape, and adaptive
neuronal mechanisms. We find that initialization and surrogate gradient width
play a crucial part in enabling learning with sparse inputs, while the
inclusion of adaptivity and learnable neuronal parameters can improve
performance. We show that the performance of the proposed ANNs and SNNs are on
par with that of the current state-of-the-art ANNs trained in a self-supervised
manner.
