Neural Network Quantization with AI Model Efficiency Toolkit (AIMET)
  While neural networks have advanced the frontiers in many machine learning
applications, they often come at a high computational cost. Reducing the power
and latency of neural network inference is vital to integrating modern networks
into edge devices with strict power and compute requirements. Neural network
quantization is one of the most effective ways of achieving these savings, but
the additional noise it induces can lead to accuracy degradation. In this white
paper, we present an overview of neural network quantization using AI Model
Efficiency Toolkit (AIMET). AIMET is a library of state-of-the-art quantization
and compression algorithms designed to ease the effort required for model
optimization and thus drive the broader AI ecosystem towards low latency and
energy-efficient inference. AIMET provides users with the ability to simulate
as well as optimize PyTorch and TensorFlow models. Specifically for
quantization, AIMET includes various post-training quantization (PTQ, cf.
chapter 4) and quantization-aware training (QAT, cf. chapter 5) techniques that
guarantee near floating-point accuracy for 8-bit fixed-point inference. We
provide a practical guide to quantization via AIMET by covering PTQ and QAT
workflows, code examples and practical tips that enable users to efficiently
and effectively quantize models using AIMET and reap the benefits of low-bit
integer inference.
