Automatic low-bit hybrid quantization of neural networks through meta
  learning
  Model quantization is a widely used technique to compress and accelerate deep
neural network (DNN) inference, especially when deploying to edge or IoT
devices with limited computation capacity and power consumption budget. The
uniform bit width quantization across all the layers is usually sub-optimal and
the exploration of hybrid quantization for different layers is vital for
efficient deep compression. In this paper, we employ the meta learning method
to automatically realize low-bit hybrid quantization of neural networks. A
MetaQuantNet, together with a Quantization function, are trained to generate
the quantized weights for the target DNN. Then, we apply a genetic algorithm to
search the best hybrid quantization policy that meets compression constraints.
With the best searched quantization policy, we subsequently retrain or finetune
to further improve the performance of the quantized target network. Extensive
experiments demonstrate the performance of searched hybrid quantization scheme
surpass that of uniform bitwidth counterpart. Compared to the existing
reinforcement learning (RL) based hybrid quantization search approach that
relies on tedious explorations, our meta learning approach is more efficient
and effective for any compression requirements since the MetaQuantNet only
needs be trained once.
