Guidelines and Benchmarks for Deployment of Deep Learning Models on
  Smartphones as Real-Time Apps
  Deep learning solutions are being increasingly used in mobile applications.
Although there are many open-source software tools for the development of deep
learning solutions, there are no guidelines in one place in a unified manner
for using these tools towards real-time deployment of these solutions on
smartphones. From the variety of available deep learning tools, the most suited
ones are used in this paper to enable real-time deployment of deep learning
inference networks on smartphones. A uniform flow of implementation is devised
for both Android and iOS smartphones. The advantage of using multi-threading to
achieve or improve real-time throughputs is also showcased. A benchmarking
framework consisting of accuracy, CPU/GPU consumption and real-time throughput
is considered for validation purposes. The developed deployment approach allows
deep learning models to be turned into real-time smartphone apps with ease
based on publicly available deep learning and smartphone software tools. This
approach is applied to six popular or representative convolutional neural
network models and the validation results based on the benchmarking metrics are
reported.
