To Fold or Not to Fold: a Necessary and Sufficient Condition on
  Batch-Normalization Layers Folding
  Batch-Normalization (BN) layers have become fundamental components in the
evermore complex deep neural network architectures. Such models require
acceleration processes for deployment on edge devices. However, BN layers add
computation bottlenecks due to the sequential operation processing: thus, a
key, yet often overlooked component of the acceleration process is BN layers
folding. In this paper, we demonstrate that the current BN folding approaches
are suboptimal in terms of how many layers can be removed. We therefore provide
a necessary and sufficient condition for BN folding and a corresponding optimal
algorithm. The proposed approach systematically outperforms existing baselines
and allows to dramatically reduce the inference time of deep neural networks.
