Deep Compression for PyTorch Model Deployment on Microcontrollers
  Neural network deployment on low-cost embedded systems, hence on
microcontrollers (MCUs), has recently been attracting more attention than ever.
Since MCUs have limited memory capacity as well as limited compute-speed, it is
critical that we employ model compression, which reduces both memory and
compute-speed requirements. In this paper, we add model compression,
specifically Deep Compression, and further optimize Unlu's earlier work on
arXiv, which efficiently deploys PyTorch models on MCUs. First, we prune the
weights in convolutional and fully connected layers. Secondly, the remaining
weights and activations are quantized to 8-bit integers from 32-bit
floating-point. Finally, forward pass functions are compressed using special
data structures for sparse matrices, which store only nonzero weights (without
impacting performance and accuracy). In the case of the LeNet-5 model, the
memory footprint was reduced by 12.45x, and the inference speed was boosted by
2.57x.
