Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided
  Compression
  Deep spiking neural networks (SNNs) have emerged as a potential alternative
to traditional deep learning frameworks, due to their promise to provide
increased compute efficiency on event-driven neuromorphic hardware. However, to
perform well on complex vision applications, most SNN training frameworks yield
large inference latency which translates to increased spike activity and
reduced energy efficiency. Hence,minimizing average spike activity while
preserving accuracy indeep SNNs remains a significant challenge and
opportunity.This paper presents a non-iterative SNN training technique
thatachieves ultra-high compression with reduced spiking activitywhile
maintaining high inference accuracy. In particular, our framework first uses
the attention-maps of an un compressed meta-model to yield compressed ANNs.
This step can be tuned to support both irregular and structured channel pruning
to leverage computational benefits over a broad range of platforms. The
framework then performs sparse-learning-based supervised SNN training using
direct inputs. During the training, it jointly optimizes the SNN weight,
threshold, and leak parameters to drastically minimize the number of time steps
required while retaining compression. To evaluate the merits of our approach,
we performed experiments with variants of VGG and ResNet, on both CIFAR-10 and
CIFAR-100, and VGG16 on Tiny-ImageNet.The SNN models generated through the
proposed technique yield SOTA compression ratios of up to 33.4x with no
significant drops in accuracy compared to baseline unpruned counterparts.
Compared to existing SNN pruning methods, we achieve up to 8.3x higher
compression with improved accuracy.
