UDC: Unified DNAS for Compressible TinyML Models
  Emerging Internet-of-things (IoT) applications are driving deployment of
neural networks (NNs) on heavily constrained low-cost hardware (HW) platforms,
where accuracy is typically limited by memory capacity. To address this TinyML
challenge, new HW platforms like neural processing units (NPUs) have support
for model compression, which exploits aggressive network quantization and
unstructured pruning optimizations. The combination of NPUs with HW compression
and compressible models allows more expressive models in the same memory
footprint.
  However, adding optimizations for compressibility on top of conventional NN
architecture choices expands the design space across which we must make
balanced trade-offs. This work bridges the gap between NPU HW capability and NN
model design, by proposing a neural architecture search (NAS) algorithm to
efficiently search a large design space, including: network depth, operator
type, layer width, bitwidth, sparsity, and more. Building on differentiable NAS
(DNAS) with several key improvements, we demonstrate Unified DNAS for
Compressible models (UDC) on CIFAR100, ImageNet, and DIV2K super resolution
tasks. On ImageNet, we find Pareto dominant compressible models, which are 1.9x
smaller or 5.76% more accurate.
