SME: ReRAM-based Sparse-Multiplication-Engine to Squeeze-Out Bit
  Sparsity of Neural Network
  Resistive Random-Access-Memory (ReRAM) crossbar is a promising technique for
deep neural network (DNN) accelerators, thanks to its in-memory and in-situ
analog computing abilities for Vector-Matrix Multiplication-and-Accumulations
(VMMs). However, it is challenging for crossbar architecture to exploit the
sparsity in the DNN. It inevitably causes complex and costly control to exploit
fine-grained sparsity due to the limitation of tightly-coupled crossbar
structure. As the countermeasure, we developed a novel ReRAM-based DNN
accelerator, named Sparse-Multiplication-Engine (SME), based on a hardware and
software co-design framework. First, we orchestrate the bit-sparse pattern to
increase the density of bit-sparsity based on existing quantization methods.
Second, we propose a novel weigh mapping mechanism to slice the bits of a
weight across the crossbars and splice the activation results in peripheral
circuits. This mechanism can decouple the tightly-coupled crossbar structure
and cumulate the sparsity in the crossbar. Finally, a superior squeeze-out
scheme empties the crossbars mapped with highly-sparse non-zeros from the
previous two steps. We design the SME architecture and discuss its use for
other quantization methods and different ReRAM cell technologies. Compared with
prior state-of-the-art designs, the SME shrinks the use of crossbars up to 8.7x
and 2.1x using Resent-50 and MobileNet-v2, respectively, with less than 0.3%
accuracy drop on ImageNet.
