TOCO: A Framework for Compressing Neural Network Models Based on
  Tolerance Analysis
  Neural network compression methods have enabled deploying large models on
emerging edge devices with little cost, by adapting already-trained models to
the constraints of these devices. The rapid development of AI-capable edge
devices with limited computation and storage requires streamlined methodologies
that can efficiently satisfy the constraints of different devices. In contrast,
existing methods often rely on heuristic and manual adjustments to maintain
accuracy, support only coarse compression policies, or target specific device
constraints that limit their applicability. We address these limitations by
proposing the TOlerance-based COmpression (TOCO) framework. TOCO uses an
in-depth analysis of the model, to maintain the accuracy, in an active learning
system. The results of the analysis are tolerances that can be used to perform
compression in a fine-grained manner. Finally, by decoupling compression from
the tolerance analysis, TOCO allows flexibility to changes in the hardware.
