FactorizeNet: Progressive Depth Factorization for Efficient Network
  Architecture Exploration Under Quantization Constraints
  Depth factorization and quantization have emerged as two of the principal
strategies for designing efficient deep convolutional neural network (CNN)
architectures tailored for low-power inference on the edge. However, there is
still little detailed understanding of how different depth factorization
choices affect the final, trained distributions of each layer in a CNN,
particularly in the situation of quantized weights and activations. In this
study, we introduce a progressive depth factorization strategy for efficient
CNN architecture exploration under quantization constraints. By algorithmically
increasing the granularity of depth factorization in a progressive manner, the
proposed strategy enables a fine-grained, low-level analysis of layer-wise
distributions. Thus enabling the gain of in-depth, layer-level insights on
efficiency-accuracy tradeoffs under fixed-precision quantization. Such a
progressive depth factorization strategy also enables efficient identification
of the optimal depth-factorized macroarchitecture design (which we will refer
to here as FactorizeNet) based on the desired efficiency-accuracy requirements.
