Neural Architecture Search for Spiking Neural Networks
  Spiking Neural Networks (SNNs) have gained huge attention as a potential
energy-efficient alternative to conventional Artificial Neural Networks (ANNs)
due to their inherent high-sparsity activation. However, most prior SNN methods
use ANN-like architectures (e.g., VGG-Net or ResNet), which could provide
sub-optimal performance for temporal sequence processing of binary information
in SNNs. To address this, in this paper, we introduce a novel Neural
Architecture Search (NAS) approach for finding better SNN architectures.
Inspired by recent NAS approaches that find the optimal architecture from
activation patterns at initialization, we select the architecture that can
represent diverse spike activation patterns across different data samples
without training. Furthermore, to leverage the temporal correlation among the
spikes, we search for feed forward connections as well as backward connections
(i.e., temporal feedback connections) between layers. Interestingly, SNASNet
found by our search algorithm achieves higher performance with backward
connections, demonstrating the importance of designing SNN architecture for
suitably using temporal information. We conduct extensive experiments on three
image recognition benchmarks where we show that SNASNet achieves
state-of-the-art performance with significantly lower timesteps (5 timesteps).
