26ms Inference Time for ResNet-50: Towards Real-Time Execution of all
  DNNs on Smartphone
  With the rapid emergence of a spectrum of high-end mobile devices, many
applications that required desktop-level computation capability formerly can
now run on these devices without any problem. However, without a careful
optimization, executing Deep Neural Networks (a key building block of the
real-time video stream processing that is the foundation of many popular
applications) is still challenging, specifically, if an extremely low latency
or high accuracy inference is needed. This work presents CADNN, a programming
framework to efficiently execute DNN on mobile devices with the help of
advanced model compression (sparsity) and a set of thorough architecture-aware
optimization. The evaluation result demonstrates that CADNN outperforms all the
state-of-the-art dense DNN execution frameworks like TensorFlow Lite and TVM.
