A High-Performance Adaptive Quantization Approach for Edge CNN
  Applications
  Recent convolutional neural network (CNN) development continues to advance
the state-of-the-art model accuracy for various applications. However, the
enhanced accuracy comes at the cost of substantial memory bandwidth and storage
requirements and demanding computational resources. Although in the past the
quantization methods have effectively reduced the deployment cost for edge
devices, it suffers from significant information loss when processing the
biased activations of contemporary CNNs. In this paper, we hence introduce an
adaptive high-performance quantization method to resolve the issue of biased
activation by dynamically adjusting the scaling and shifting factors based on
the task loss. Our proposed method has been extensively evaluated on image
classification models (ResNet-18/34/50, MobileNet-V2, EfficientNet-B0) with
ImageNet dataset, object detection model (YOLO-V4) with COCO dataset, and
language models with PTB dataset. The results show that our 4-bit integer
(INT4) quantization models achieve better accuracy than the state-of-the-art
4-bit models, and in some cases, even surpass the golden full-precision models.
The final designs have been successfully deployed onto extremely
resource-constrained edge devices for many practical applications.
