EAST: Encoding-Aware Sparse Training for Deep Memory Compression of
  ConvNets
  The implementation of Deep Convolutional Neural Networks (ConvNets) on tiny
end-nodes with limited non-volatile memory space calls for smart compression
strategies capable of shrinking the footprint yet preserving predictive
accuracy. There exist a number of strategies for this purpose, from those that
play with the topology of the model or the arithmetic precision, e.g. pruning
and quantization, to those that operate a model agnostic compression, e.g.
weight encoding. The tighter the memory constraint, the higher the probability
that these techniques alone cannot meet the requirement, hence more awareness
and cooperation across different optimizations become mandatory. This work
addresses the issue by introducing EAST, Encoding-Aware Sparse Training, a
novel memory-constrained training procedure that leads quantized ConvNets
towards deep memory compression. EAST implements an adaptive group pruning
designed to maximize the compression rate of the weight encoding scheme (the
LZ4 algorithm in this work). If compared to existing methods, EAST meets the
memory constraint with lower sparsity, hence ensuring higher accuracy. Results
conducted on a state-of-the-art ConvNet (ResNet-9) deployed on a low-power
microcontroller (ARM Cortex-M4) validate the proposal.
