Quantization and Deployment of Deep Neural Networks on Microcontrollers
  Embedding Artificial Intelligence onto low-power devices is a challenging
task that has been partly overcome with recent advances in machine learning and
hardware design. Presently, deep neural networks can be deployed on embedded
targets to perform different tasks such as speech recognition,object detection
or Human Activity Recognition. However, there is still room for optimization of
deep neural networks onto embedded devices. These optimizations mainly address
power consumption,memory and real-time constraints, but also an easier
deployment at the edge. Moreover, there is still a need for a better
understanding of what can be achieved for different use cases. This work
focuses on quantization and deployment of deep neural networks onto low-power
32-bit microcontrollers. The quantization methods, relevant in the context of
an embedded execution onto a microcontroller, are first outlined. Then, a new
framework for end-to-end deep neural networks training, quantization and
deployment is presented. This framework, called MicroAI, is designed as an
alternative to existing inference engines (TensorFlow Lite for Microcontrollers
and STM32CubeAI). Our framework can indeed be easily adjusted and/or extended
for specific use cases. Execution using single precision 32-bit floating-point
as well as fixed-point on 8- and 16-bit integers are supported. The proposed
quantization method is evaluated with three different datasets (UCI-HAR, Spoken
MNIST and GTSRB). Finally, a comparison study between MicroAI and both existing
embedded inference engines is provided in terms of memory and power efficiency.
On-device evaluation is done using ARM Cortex-M4F-based microcontrollers (Ambiq
Apollo3 and STM32L452RE).
