BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer
  As the applications of deep learning models on edge devices increase at an
accelerating pace, fast adaptation to various scenarios with varying resource
constraints has become a crucial aspect of model deployment. As a result, model
optimization strategies with adaptive configuration are becoming increasingly
popular. While single-shot quantized neural architecture search enjoys
flexibility in both model architecture and quantization policy, the combined
search space comes with many challenges, including instability when training
the weight-sharing supernet and difficulty in navigating the exponentially
growing search space. Existing methods tend to either limit the architecture
search space to a small set of options or limit the quantization policy search
space to fixed precision policies. To this end, we propose BatchQuant, a robust
quantizer formulation that allows fast and stable training of a compact,
single-shot, mixed-precision, weight-sharing supernet. We employ BatchQuant to
train a compact supernet (offering over $10^{76}$ quantized subnets) within
substantially fewer GPU hours than previous methods. Our approach,
Quantized-for-all (QFA), is the first to seamlessly extend one-shot
weight-sharing NAS supernet to support subnets with arbitrary ultra-low
bitwidth mixed-precision quantization policies without retraining. QFA opens up
new possibilities in joint hardware-aware neural architecture search and
quantization. We demonstrate the effectiveness of our method on ImageNet and
achieve SOTA Top-1 accuracy under a low complexity constraint ($<20$ MFLOPs).
The code and models will be made publicly available at
https://github.com/bhpfelix/QFA.
