Meta-learning Spiking Neural Networks with Surrogate Gradient Descent
  Adaptive "life-long" learning at the edge and during online task performance
is an aspirational goal of AI research. Neuromorphic hardware implementing
Spiking Neural Networks (SNNs) are particularly attractive in this regard, as
their real-time, event-based, local computing paradigm makes them suitable for
edge implementations and fast learning. However, the long and iterative
learning that characterizes state-of-the-art SNN training is incompatible with
the physical nature and real-time operation of neuromorphic hardware. Bi-level
learning, such as meta-learning is increasingly used in deep learning to
overcome these limitations. In this work, we demonstrate gradient-based
meta-learning in SNNs using the surrogate gradient method that approximates the
spiking threshold function for gradient estimations. Because surrogate
gradients can be made twice differentiable, well-established, and effective
second-order gradient meta-learning methods such as Model Agnostic Meta
Learning (MAML) can be used. We show that SNNs meta-trained using MAML match or
exceed the performance of conventional ANNs meta-trained with MAML on
event-based meta-datasets. Furthermore, we demonstrate the specific advantages
that accrue from meta-learning: fast learning without the requirement of high
precision weights or gradients. Our results emphasize how meta-learning
techniques can become instrumental for deploying neuromorphic learning
technologies on real-world problems.
