Tiny but Accurate: A Pruned, Quantized and Optimized Memristor Crossbar
  Framework for Ultra Efficient DNN Implementation
  The state-of-art DNN structures involve intensive computation and high memory
storage. To mitigate the challenges, the memristor crossbar array has emerged
as an intrinsically suitable matrix computation and low-power acceleration
framework for DNN applications. However, the high accuracy solution for extreme
model compression on memristor crossbar array architecture is still waiting for
unraveling. In this paper, we propose a memristor-based DNN framework which
combines both structured weight pruning and quantization by incorporating
alternating direction method of multipliers (ADMM) algorithm for better pruning
and quantization performance. We also discover the non-optimality of the ADMM
solution in weight pruning and the unused data path in a structured pruned
model. Motivated by these discoveries, we design a software-hardware
co-optimization framework which contains the first proposed Network
Purification and Unused Path Removal algorithms targeting on post-processing a
structured pruned model after ADMM steps. By taking memristor hardware
constraints into our whole framework, we achieve extreme high compression ratio
on the state-of-art neural network structures with minimum accuracy loss. For
quantizing structured pruned model, our framework achieves nearly no accuracy
loss after quantizing weights to 8-bit memristor weight representation. We
share our models at anonymous link https://bit.ly/2VnMUy0.
