BioGrad: Biologically Plausible Gradient-Based Learning for Spiking
  Neural Networks
  Spiking neural networks (SNN) are delivering energy-efficient, massively
parallel, and low-latency solutions to AI problems, facilitated by the emerging
neuromorphic chips. To harness these computational benefits, SNN need to be
trained by learning algorithms that adhere to brain-inspired neuromorphic
principles, namely event-based, local, and online computations. Yet, the
state-of-the-art SNN training algorithms are based on backprop that does not
follow the above principles. Due to its limited biological plausibility, the
application of backprop to SNN requires non-local feedback pathways for
transmitting continuous-valued errors, and relies on gradients from future
timesteps. The introduction of biologically plausible modifications to backprop
has helped overcome several of its limitations, but limits the degree to which
backprop is approximated, which hinders its performance. We propose a
biologically plausible gradient-based learning algorithm for SNN that is
functionally equivalent to backprop, while adhering to all three neuromorphic
principles. We introduced multi-compartment spiking neurons with local
eligibility traces to compute the gradients required for learning, and a
periodic "sleep" phase to further improve the approximation to backprop during
which a local Hebbian rule aligns the feedback and feedforward weights. Our
method achieved the same level of performance as backprop with multi-layer
fully connected SNN on MNIST (98.13%) and the event-based N-MNIST (97.59%)
datasets. We deployed our learning algorithm on Intel's Loihi to train a
1-hidden-layer network for MNIST, and obtained 93.32% test accuracy while
consuming 400 times less energy per training sample than BioGrad on GPU. Our
work shows that optimal learning is feasible in neuromorphic computing, and
further pursuing its biological plausibility can better capture the benefits of
this emerging computing paradigm.
