Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking
  Neural Networks
  Spiking neural networks (SNNs) are positioned to enable spatio-temporal
information processing and ultra-low power event-driven neuromorphic hardware.
However, SNNs are yet to reach the same performances of conventional deep
artificial neural networks (ANNs), a long-standing challenge due to complex
dynamics and non-differentiable spike events encountered in training. The
existing SNN error backpropagation (BP) methods are limited in terms of
scalability, lack of proper handling of spiking discontinuities, and/or
mismatch between the rate-coded loss function and computed gradient. We present
a hybrid macro/micro level backpropagation (HM2-BP) algorithm for training
multi-layer SNNs. The temporal effects are precisely captured by the proposed
spike-train level post-synaptic potential (S-PSP) at the microscopic level. The
rate-coded errors are defined at the macroscopic level, computed and
back-propagated across both macroscopic and microscopic levels. Different from
existing BP methods, HM2-BP directly computes the gradient of the rate-coded
loss function w.r.t tunable parameters. We evaluate the proposed HM2-BP
algorithm by training deep fully connected and convolutional SNNs based on the
static MNIST [14] and dynamic neuromorphic N-MNIST [26]. HM2-BP achieves an
accuracy level of 99.49% and 98.88% for MNIST and N-MNIST, respectively,
outperforming the best reported performances obtained from the existing SNN BP
algorithms. Furthermore, the HM2-BP produces the highest accuracies based on
SNNs for the EMNIST [3] dataset, and leads to high recognition accuracy for the
16-speaker spoken English letters of TI46 Corpus [16], a challenging
patio-temporal speech recognition benchmark for which no prior success based on
SNNs was reported. It also achieves competitive performances surpassing those
of conventional deep learning models when dealing with asynchronous spiking
streams.
