Theory and Tools for the Conversion of Analog to Spiking Convolutional
  Neural Networks
  Deep convolutional neural networks (CNNs) have shown great potential for
numerous real-world machine learning applications, but performing inference in
large CNNs in real-time remains a challenge. We have previously demonstrated
that traditional CNNs can be converted into deep spiking neural networks
(SNNs), which exhibit similar accuracy while reducing both latency and
computational load as a consequence of their data-driven, event-based style of
computing. Here we provide a novel theory that explains why this conversion is
successful, and derive from it several new tools to convert a larger and more
powerful class of deep networks into SNNs. We identify the main sources of
approximation errors in previous conversion methods, and propose simple
mechanisms to fix these issues. Furthermore, we develop spiking implementations
of common CNN operations such as max-pooling, softmax, and batch-normalization,
which allow almost loss-less conversion of arbitrary CNN architectures into the
spiking domain. Empirical evaluation of different network architectures on the
MNIST and CIFAR10 benchmarks leads to the best SNN results reported to date.
