S2Engine: A Novel Systolic Architecture for Sparse Convolutional Neural
  Networks
  Convolutional neural networks (CNNs) have achieved great success in
performing cognitive tasks. However, execution of CNNs requires a large amount
of computing resources and generates heavy memory traffic, which imposes a
severe challenge on computing system design. Through optimizing parallel
executions and data reuse in convolution, systolic architecture demonstrates
great advantages in accelerating CNN computations. However, regular internal
data transmission path in traditional systolic architecture prevents the
systolic architecture from completely leveraging the benefits introduced by
neural network sparsity. Deployment of fine-grained sparsity on the existing
systolic architectures is greatly hindered by the incurred computational
overheads. In this work, we propose S2Engine $-$ a novel systolic architecture
that can fully exploit the sparsity in CNNs with maximized data reuse. S2Engine
transmits compressed data internally and allows each processing element to
dynamically select an aligned data from the compressed dataflow in convolution.
Compared to the naive systolic array, S2Engine achieves about $3.2\times$ and
about $3.0\times$ improvements on speed and energy efficiency, respectively.
