{'fraction_non_alphanumeric': 0.025682310088159216, 'fraction_numerical': 0.004460750673071847, 'mean_word_length': 5.096057924376509, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deep Learning on Mobile Devices - A Review

Recent breakthroughs in deep learning and artificial intelligence technologies have enabled numerous mobile applications. While traditional computation paradigms rely on mobile sensing and cloud computing, deep learning implemented on mobile devices provides several advantages. These advantages include low communication bandwidth, small cloud computing resource cost, quick response time, and improved data privacy. Research and development of deep learning on mobile and embedded devices has recently attracted much attention. This paper provides a timely review of this fast-paced field to give the researcher, engineer, practitioner, and graduate student a quick grasp on the recent advancements of deep learning on mobile devices. In this paper, we discuss hardware architectures for mobile deep learning, including Field Programmable Gate Arrays, Application Specific Integrated Circuit, and recent mobile Graphic Processing Units. We present Size, Weight, Area and Power considerations and their relation to algorithm optimizations, such as quantization, pruning, compression, and approximations that simplify computation while retaining performance accuracy. We cover existing systems and give a state-of-the-industry review of TensorFlow, MXNet, Mobile AI Compute Engine, and Paddle-mobile deep learning platform. We discuss resources for mobile deep learning practitioners, including tools, libraries, models, and performance benchmarks. We present applications of various mobile sensing modalities to industries, ranging from robotics, healthcare and multi-media, biometrics to autonomous drive and defense. We address the key deep learning challenges to overcome, including low quality data, and small training/adaptation data sets. In addition, the review provides numerous citations and links to existing code bases implementing various technologies.


INTRODUCTION
Deep learning is the key enabler for many recent advances in artificial intelligence applications. The greater benefit will come when the artificial intelligence technologies becomes ubiquitous in mobile applications, such as automatous driving, affordable robots for home, and more intelligent personal assistance on mobile phone. Compared with the traditional mobile sensing and cloud computing paradigm, the advantages of deep learning and inference on the mobile device are four folds: 1) The saving of communication bandwidth. The more computing done on the mobile device, the less data needs to be sent to the cloud. 2) The reduction of cloud computing resource cost. The cost of maintaining or even renting cloud computing resources can be prohibitive for some applications. Computation on mobile devices becomes possible as mobile devices become more computationally powerful. 3) The quick response time. If all computation can be performed locally, there will be no overhead in communication time or any concerns of server reliability. For some applications, such as in the health care and military domains, this response time is very critical. 4) Mobile computing keeps the sensory data on the local device, thus greatly improving user data privacy. This is particularly true for home robotics applications.
Research in deep learning for mobile and embedded devices has thus becomes a very hot topic. The subtopics cover hardware architecture considerations, algorithmic optimizations, and choices for mobile deep learning platforms. As the research and development progresses, many tools, data, and models are made publicly available. For a beginner to mobile deep learning, the amount of information available in this topic can be huge. This paper gives the reader a solid grasp on these topics enabling them to quickly start to make contributions to this fast growing field. This is achieved by reviewing and comparing existing hardware, software, algorithmic, and platform choices, listing available resources, guidance in making mobile deep learning applications, and discussing the challenging problems to be solved. This paper reviews recent scientific and technological advances to bring deep learning to mobile and embedded devices. However, it does not cover the basics of machine learning (ML) and deep learning. Readers should refer to more basic tutorials in deep learning if necessary, e.g. [29] [30]. This paper begins with the hardware aspects for deep learning. The hardware architecture design tradeoffs for system accuracy, power consumption, and cost will be discussed in Section 2. A state-of-the-industry review will be given in Section 3 to detail some of the most promising mobile deep learning computation platforms and libraries. The pros, cons, and application domains of each platform will be discussed. It details potential development paths based on the timeframe available to the developer. Different paths are available given development timeframes from hours, days, weeks, to months. The software and algorithmic advancement for mobile deep learning are described in Section 4. Optimizations of deep learning algorithms, such as quantization, pruning, compression, and approximation have to be made while retaining as much performance accuracy as possible. For the convenience of the mobile deep learning practitioners, Section 5 provides a list of tools, models, and some publicly available codebases to start with. These resource significantly lower the barriers of entry for beginners. For readers interested in developing real mobile deep learning applications, Section 6 showcases some recent successes in mobile deep learning development in various application domains, ranging from robotics, autonomous driving, healthcare, multi-media, biometrics, entertainment and defense. Although a great deal of resources and progress has been made for mobile deep learning, challenges remain. Section 7 outlines a few key challenges, which could serve as important research topics for graduate students.

HARDWARE ARCHITECTURES FOR MOBILE DEEP LEARNING
Advances in deep learning are mostly attributed to the availability of powerful computation resources and large amounts of training data. One of the key contributors to increased computation power comes from low cost GPUs. A high performance deep learning application requires a high quality machine learning model trained with a large amount of data, often on the order of terabytes. This is typically achieved using high performance Central CPUs, GPUs, Tensor Processing Units (TPUs), or their combination. Once the model is built, it can be deployed on a mobile device with much less computation power, including CPU, ASIC, FPGAs, and mobile GPUs. For mobile deep learning applications, the computation hardware architecture decisions are based on the computation performance, system SWAP, and cost (in term of time and money for the development) considerations.

CPU
Although CPUs can be cumbersome for building state-of-the-art deep learning models when the training data is very large, they are a valid option for training or adapting models to small amounts of data and to deploy pre-built deep learning models. The obvious advantage of deploying mobile deep learning on a CPU is that CPUs are available for most tablets and mobile phones. In addition, these devices often come with powerful CPUs and have a large selection of built-in sensors. This means there is no extra hardware cost to enable more AI applications for a potential huge market. Real-time applications can be realized on a standard CPU with little or no extra effort. However, a full grasp of a few important techniques can be very beneficial to building computation-and power-efficient applications for CPUs.
• Use the highly optimized linear algebra library LAPACK/BLAS, most existing deep learning frameworks already take this into account [1]; • Take full advantage of multi-core multi-threaded CPUs for computation parallelization; • Make use of CPU manufacturers-specific mathematic library; • Conduct tradeoff studies of model size, numeric precision, computation speed, and system performance.
Many successful deep learning applications have been implemented on CPUs, including voice, fingerprint, and face biometrics on mobile devices. The market has driven the mobile technology to a point where the CPUs on mainstream mobile devices are just as powerful as CPUs on mainstream tablets and laptops.

GPUs and Mobile GPUs
The GPU is a highly parallel processor by physical design. It typically contains thousands of repetitive small cores, making it much more efficient to do certain types of repetitive computations on matrices than general purpose CPUs. The GPU was originally designed mostly for graphics and gaming, but it became the key enabler for building high performance machine learning models in the past few years. Now, GPUs are widely used in embedded systems, mobile phones, PCs, workstations, and game consoles. NVIDIA offers a wide range of GPU products. The NVIDIA GPUs are very useful for machine learning engineers, as they provides a powerful programming platform and application programming interface (API) called Compute Unified Device Architecture (CUDA) [3]. CUDA works with a few of the most popular programming languages, such as Python, C, C++, and FORTRAN. For example, pyCUDA makes it much easier for a Python programmer to utilize the GPU resources. Some of the most popular NVIDIA GPUs relevant to deep learning are listed in Table 1. The performance is measured in terms of double precision TFLOPS, the number of tera floating point operation per second. For the serious deep learning researcher, a GPU with performance at the level of GTX 1080 Ti or higher should be chosen to build state of the art models when large amounts of training data are available. This GPU provides 332 TFLOPS computation power. Many supercomputers are made of Tesla GPUs, while the Drive PX GPUs are popular system-on-a-chip component for autonomous driving [31].  [32]. A list of GPUs made for androids is listed in [33]. One of the most notable smartphone system on a chip (SoC) is the Qualcomm Snapdragon chip [34]. The SoC features multi-core CPU, a GPU, GPS, wireless modem, among others, making many applications of deep learning on a smartphone feasible.

FPGA
While the CPU is made for extremely general purpose computation, and an ASIC is almost made exclusively for a specific application, the FPGA lies in between. FPGAs can be (re)programmed by 'firmware' to perform many specific applications very efficiently. It also lies in between in terms of system development time and power consumption. At the basic level, FPGAs use flip-flop circuits to implement sequential logic functions and table look-ups, i.e. memory circuit to implement combinational logic function. The logic functions are implemented by programming memory, which also controls switch circuit connections, thus FPGAs do not need to explicitly perform the logic operation once it is programmed. Modern FPGAs tend to take a SoC approach to integrate process core, communication core, and memory on a single chip.
FPGA venders, such as Xilinx and Altera, have made many software tools to make FPGA programming easier. While traditional FPGA programmers need to grasp knowledge of digital circuits and hardware description language (HDL), the trend of FPGA tools are towards high-level synthesis of circuits (HLS) [40].

ASIC and TPU
The ASIC goes one step further than the FPGA to design and fabricate specific chips for an application. It has the potential to achieve even better SWAP, but at the cost of long development time. Building ASICs for mobile machine learning and artificial intelligence has a very long history. It was called neuromorphic engineering in the early days [38] [39]. Many analog, digital, mixed-signal, memristor, and spike circuits are designed to mimic auditory, vision, and brain functions [41] [42].
There are many ASICs developed for AI applications. TPU is one developed specifically for neural network machine learning by Google. It was used in the recent AlphaGo vs Lee Sedol man-machine Go games. Although not commercially available, TPU is offered to the machine learning researcher for free trial and is accessible through its cloud computing service. TPU provides 10s~100s TFLOPS and can fit into the rack slot of common cloud computer servers.

Comparison of CPU, GPU, FPGA, and ASIC for Mobile Deep Learning
As mentioned in previous subsections, the CPU and GPU are general purpose computing platforms, thus offering the most flexibility. Early algorithmic performance study of a deep learning application should make use of CPUs and GPUs to get a preliminary idea of achievable performance. CPUs and GPUs support full precision computing and can afford computation intensive models, which often means higher prediction accuracy is achievable. However, GPUs and CPUs are less power efficient. ASICs can be much more energy efficient as the hardware is made specifically for a certain computation. Yet, the design and development of ASIC chips can be very time consuming. An ASIC is thus only used when the algorithm research is settled and the system has a very lower power budget. FPGAs offers a tradeoff among power consumption, prediction accuracy, and system development speed. The typical achievable power consumption and ML accuracy metrics for these four computation architectures are illustrated in Figure 1

MOBILE DEEP LEARNING PLATFORMS AND LIBRARIES
In the past few years, many platforms have been developed by the major internet companies for deep learning application on mobile devices. Depending on time to market constraints, different platforms should be chosen. Choosing the right platform can make a significant difference in development time, cost, and final system performance. The following subsection describes approaches with development time ranging from hours to years.

Building Mobile Deep Learning Solutions Leveraging Cloud AI services
If deep learning on the cloud is appropriate for the application, the quickest way to deploy deep learning capability on a mobile device is to take advantage of many existing cloud artificial intelligence APIs. In this case, the mobile device serves as a sensor and user interface. These APIs offer ready-to-use AI capabilities in machine learning, speech recognition, computer vision, natural language processing, AI assistant, knowledge discovery, personality and emotion analysis, and search. Some of these well-known services include Microsoft Cognitive Services, Google Cloud Vision, IBM Watson Services, Amazon Rekognition, and Baidu EZDL. If the application to be developed can use existing welltested APIs, than this use can dramatically reduce development cost.
For developers with sufficient experience with existing cloud AI service APIs, the development timeframe can be as short as hours or days. However, it may be that existing APIs may not offer the desired capability, or performance may not be fully satisfactory on the data. In that case, deeper investigation is needed to leverage lower level existing tools, models, data, detailed in the following subsection.

Building Mobile Deep Learning Solutions Leveraging Existing Platforms
Some deep learning applications, or at least the inference part, needs to be implemented on mobile devices without relying on cloud machine learning.

Keras
Some deep machine learning toolkits, such as TensorFlow, require the user to have very good knowledge of both machine learning and software engineering. Users with both skillsets are very limited, thus the adoption of ML was slow. The idea of Keras is to greatly simplify the learning and deployment of DL. By designing an easy to use highlevel API and hide all unnecessary machine learning and software details, Keras python enables users to use TensorFlow, CNTK, or Theano without going through the learning curve to study each underlying library in detail. Keras code and documentation are available at https://keras.io/. The Keras project is so successful that it is fully integrated into TensorFlow now. Keras is one of the most popular tool for beginners and researchers in deep learning.
There are numerous open source deep learning projects implemented in Keras available on github.

Torch/PyTorch
PyTorch is a Python open source ML library based on Torch. Torch itself is an open source tool for machine learning and scientific computing based on Lua programming language. Although Torch is no longer in active development, PyTorch is very popular among academic researchers. There are also many open source projects and AI applications designed based on PyTorch. Pytorch is available at https://pytorch.org/.

MXNet
MXNet from Apache Software Foundation. MXNet is a flexible and efficient library for deep learning. It packs all code in a single source file. The advantage is cross platform (iOS and android), easy porting, and usable in most major programming languages. However, only CPU is supported for deployment. MXNet is available at https://mxnet.incubator.apache.org.

CNNDroid
CNNDroid is an open source library for speed up execution of CNN

JaveScript Libraries
JaveScript is one of the core programming language for web-based application. For developers with sufficient experience with existing platforms and libraries, the development of a mobile deep learning prototype can take from a few weeks or a few months. The next step is to test the application in the field. The real tests may find the computation, storage, memory footprint, or SWAP aspects need to be further improved. In particular, as the system gathers more domain specific data, you may be able to improve the deep learning system performance. The ML designer needs to have even deeper knowledge of deep learning algorithms and software to push its performance to a higher level. The next section summarizes some progress in mobile deep learning algorithms to further improve performance in these aspects.

Conversion of models trained on different platforms and libraries
Each set of deep machine learning libraries and tools has its own advantages, disadvantages, and use cases. The models trained using these tools all have their own format. To make the best use of all these tools and libraries, it may be beneficial to be able to convert the model format from one to the other. Thus there are a few projects developed for this purpose. One of the most powerful tools for model conversion is MMdnn. This tool can convert TensorFlow, CNTK, Keras, Caffe, PyTorch, MXNet, and CoreML formats model into an intermediate representation, which can then be converted into one of these native target format [28].

OPTIMIZATION FOR MOBILE DEEP LEARNING
Efforts have been taken in the domain of software, algorithmic, and joint software and hardware co-design to optimize the SWAP performance of mobile deep learning applications. This section describes these efforts.

DeepX Software Accelerator
One of the key requirements of deep learning on mobile devices is lower resource requirements, including memory, storage, computation, and power consumption. In addition to the progress on hardware design, there is also progress made on software approaches. DeepX is such a software accelerator designed for low power deep learning inference on mobile devices [43]. It consists of two resource control algorithms for deep learning inference. One algorithm decomposes the DNN architecture into blocks of various types, which are more efficiently executed using heterogeneous local GPU and CPU processors. The other algorithm performs resource scaling to adjust DNN model architecture, thus the overhead of each block.

Algorithmic optimizations for SWAP
The original motivation for going deeper in neural networks for machine learning is for better representation learning. These learned representations capture complex non-linear embedding in the feature space and grant the model better accuracy and better generalization performance. These models are often deep, contain millions of parameters, and often not ideal for mobile deep learning deployment. Recently, some very interesting work was done to make these high accuracy models more suitable for mobile applications with little or no sacrifice in accuracy. This section summarizes some of these ideas, including quantization, pruning, compression, and approximations of DNNs.

CNN Architecture Optimization
Recent ResNet enabled training of DNN with over hundreds of layers achieved impressive performance in accuracy. However, the accuracy performance does not scale linearly with the number of layers. For mobile applications, a good trade off can be made to pick the architecture to balance accuracy and other SWAP considerations. The SqueezeNet work of Iandola proposed to optimize CNN for fewer parameters using three strategies [4]: • Replace majority of the 3x3 filters with 1x1 filter.
• Decrease the number of input channels to the filters

• Down sample in later layers in the network
The SqueezeNet is able to achieve AlexNet level accuracy on ImageNet task with 50 times few parameters. With additional model compression techniques, detailed in subsections below, the SqueezeNet achieved 0.5MB size, or over 500 times smaller than AlexNet.

Pruning, Quantization, and Huffman Coding
The "deep compression" work by Han used techniques of pruning, quantization and Huffman coding to reduce model size by a factor of 35~49 without affecting accuracy [44]. The pruning basically removes all small-weight connections. Typically, the weight parameters can be reduced by a factor of 10 using pruning. The next steps are quantization and weight sharing. The idea is to cluster all the weights with similar values together and store these weights in a codebook. The last step applies the well-known Huffman coding to assign shorter code to more common symbols.
It is also a common practice to reduce precision from 32 bits to 16 bits or less for mobile applications. Different approximation methods, such as fixed point approximation, dynamic fixed point approximation, minifloat approximation, and multiplier-free arithmetic are compared in the thesis work of Gysel [45]. An approximation framework, called Ristretto, was proposed with open-source code. TensorFlow also has 8 bit quantization support.

Network binarization and XNOR-Networks
An extreme case of quantization is binarization, i.e, the weights are reduced to either -1 or +1. The convolution computation basically turns into summation and subtraction. In XNOR network, both the filter and input to convolution layers are binary [46]. Compared with traditional 32 bit resolution computation, this results in a factor of 32 in memory savings and 58 times faster convolution operation. When evaluated on ImageNet classification, the binary weight network version of AlexNet achieved the same accuracy as the full-precision AlexNet. The authors also made the code available.

Software and Hardware Co-design
The previous subsection has shown great promise of algorithmic advances for mobile deep learning SWAP improvement. Even greater improvements can be achieved by taking an algorithmic and hardware co-design approach to make ASIC chips for mobile deep learning. For example, recent ASIC co-design work by Han, Liu, etc. has shown over four and three orders of magnitude improvement in energy efficiency than CPU and GPU, respectively. It also achieved over two and one order of magnitude improvement in speed than CPU and GPU, respectively on nine DNN benchmarks [47].

Improved Mobile Deep Learning System Performance using Domain Specific Data
The recent advances in deep machine learning algorithms have enabled many useful mobile deep learning applications developed for real use. However, for deep machine learning, the data can be just as important as the algorithm. It is not uncommon that the effort spent for collecting the right data is more rewarding than effort spent on improving the algorithm. The following subsection give some guidance on what to do with domain specific data.
Although there is a huge amount of publicly available datasets, mobile deep learning application may operate in different environments, use different sensors, or have different specifications. It is always a good idea to collect some real data from the kind of environment that the mobile deep learning application will be deployed. This data will be invaluable for improving the model or verifying the application performance. However, it is not always the best approach to build the model from scratch using new data. For mobile deep learning, there is less control of the environment, thus the testing data can have great variability. This means that to have a good performing model, a huge training dataset may be necessary. Unfortunately, data collection and data labelling in particular, can be very time consuming. Taking all these into consideration, the typical steps to proceed are: 1. Searching for pre-trained models similar to the application. For typical AI applications in computer vision, speech recognition, and NLP, there are many high quality models already made publicly available. Please refer to section 5.4 for a detailed list of existing AI application models. 2. Fine tune pre-trained models on domain specific data. 3. Leverage existing frameworks to deploy application.
Making use of existing models and frameworks can save a great deal of effort. However, deciding which part of the pretrained model to be fine-tuned can be a work of art. This depends on the amount of domain specific data acquired and the similarity between this data and the original data used to build the pre-trained model. A general guide line is given in the Table 3 [4]. Low Train a linear classifier on lower layer activation, Higher layers are more data specific to original data Large Low Train DNN from scratch using domain specific data As mentioned in this section, there is a great deal of progress made in deep learning algorithms and models for mobile applications. The next section gives more detailed list of resource available in publication, libraries, tool, data, and models.

Computer Languages for Mobile Deep Learning
Some of the most popular computer languages used in machine learning include Python, Matlab, C++, Java, R, and CUDA. Different approaches to implement mobile deep learning requires different platform, tools, and programming languages. These languages offer different development efficiencies and actual computation efficiencies. Thus, different approaches may be taken at different stages of the development. In addition, to deploy the developed system on different hardware platform, additional computer language skills are required. For example, Android and iOS platforms use different computer languages. There are also tools/computer language developed to offer solutions applicable to multiple platforms.

Open source code base for Deep Learning on Mobile Devices
The field of mobile deep learning has a relative low barrier to entry as most researchers make their source code freely available online. The most popular website to host the source code is github. For example, SqueezNet and model compression work mentioned in previous subsections are available at https://github.com/DeepScale/SqueezeNet and https://github.com/topics/model-compression. Facebook AI Research has recently open sourced Detectron platform for object detection. The trained models can be deployed on the mobile devices using Caffe2 runtime. The Detectron is available at https://github.com/facebookresearch/Detectron. In other cases, multiple versions of the same algorithm based on different platforms or libraries are made available by the machine learning community. In these case, it is usually a good idea to compare ratings, and how well documented and maintained the algorithms are.

Deep learning model zoo
As mentioned in the previous section, there are many deep learning models made publicly available. These model offer a good starting point to enable you to develop mobile deep learning applications. A few of these model zoos are listed below.
• TensorFlow/Keras model zoo. This is arguably the largest model zoo. It contain two categories, one for TensorFlow official models, and one for TensorFlow research models. The official models use TensorFlow's high-level API and tends to be better maintained and tested to keep up to date with latest TensorFlow updates. To fit a model downloaded from the zoo to a specific mobile platform, some of the previously mentioned technologies should be applied to optimize its SWAP metrics. Its performance should be benchmarked for mobile application.

Benchmarking Mobile Deep Learning Platform Performance
Modern Smartphones are capable of providing a computer power on the order of over 10 billion FLOPs. The benchmark of mobile deep learning should consider metrics such as accuracy, model size, and speed/execution time. . One benchmark is to compare various deep learning models for image classification on iPhone 7 [6] [7]. The performance is summarized in Table 4. The second one is for AI benchmark on four chipsets (Qualcomm, HiSillicon, MediaTck, and Samsung) on Android smartphones [8]. The paper compared eight tasks in deep learning as AI benchmarks, including image classification, face recognition, deblurring, super-resolution, segmentation, and enhancement. Results are obtained with this benchmark from over 10,000 mobile devices and more than 50 different mobile SoCs. Detailed comparisons are reported at http://aibenchmark.com/ . Some of the conclusions are: • The easiest way to using deep learning on Android is to use TensorFlow Mobile framework • TensorFlow Lite is an option, but we recommended for tasks more complicated than image classification • For specific device or SoC, proprietary SDK can be used, but it is not as easy and convenient • Caffe2 and other frameworks are much less popular with almost no tutorial and problem descriptions • The applicability of model quantization is limited, performance may not be reliable

Robotics
Robotics is being used increasingly in manufacturing industries, warehouse, agriculture, human access denied areas, and health care. Its use in the home is mostly limited to floor cleaning. As the AI technology progresses, robotics has many roles to play in the near future as envisioned by many scientists and engineers. In addition to the speech, NLP, and computer vision tasks mentioned in previous sections, robotics faces a few application specific tasks of learning, reasoning, and embodiment. Some recent reviews of the state of art and discussion of its potential and limitations are available [9] [10]. One key technique needed to make progress is to enable robots to autonomously acquire skills from sensory data. The robots need to be able to perform a general task without given specific instruction. This is also called learning to learn.

Autonomous driving
The advantages of Autonomous driving are numerous and can bring true revolution to transportation and life style in general. Autonomous driving is very popular in the industry as more capable companies are doing road testing of their own vehicles. Deep learning has revolutionized the perception modules for autonomous driving. Usually the perception is achieved by multi-modal sensors (e.g. EO camera, LiDARs, and Radras). Therefore, how to fuse multiple sensors in deep perception pipeline and achieve a nice model accuracy and computational cost trade-off is a very important and yet open topic. A very good discussion on this multi-modal fusion topic can be find here [53]. Many other interesting papers on applications of deep learning for autonomous driving have been published recently [11][12] [13]. An empirical evaluation of CNN on highway driving has shown its capabilities of land and vehicle detection in real time [11] [13]. A MultiNet has been proposed for real time solving of joint classification, detection, and semantic segmentation via a unified architecture [14]. Autonomous driving is another very important application of mobile deep learning that has a high hopes to become reality in the very near future. The obstacle to autonomous driving appears to be in driving in extreme conditions, in addition to non-technical issues such as ethical and legal.

Healthcare
While the health care resources may be limited, smartphone devices are increasingly available globally.. Applications of deep learning to improve the healthcare services has a potentially profound impact on well-being and the economy. With so many built-in sensors, Smartphones can be used as "medical" devices to monitor the physical and psychological status of individuals. For example, detection of depression is possible using audio and text in conversation [21]. In addition, the camera and speaker can serve as eyes and voice for people with vision and speech disorders. A special issue of AI for mobile health data analysis and processing to cover these topics are contained in [15][16] [17].

Biometrics
Similar to many other fields of pattern recognition problems, deep learning is the leading solution for many biometric recognition modalities, including face, voice, keystroke, fingerprint, finger vein, iris, and gesture recognition [18][19] [20]. As the smartphone contains so much user-private data, security is of high importance. Most smartphones have built-in fingerprint and or face recognition biometrics. As we can envision, for future shareable autonomous driving vehicles, the mobile biometrics component should also be a must for security purposes. To some extent, biometrics is an almost solved problem for cooperative subjects in close distance. One of key unsolved problem is for uncooperative subjects at a long standoff distance [18].

Personal Assistance, Mobile Multi-Media, Augmented Reality, and Entertainment
Personal assistance, mobile multi-media and people interaction are arguably the most applied domains of mobile deep learning technologies [22]. A recent report suggests over 47 million U.S. adults already have access to smart speaker, not to mention the personal assistance function available on the smartphone. Face detection is one well-known function on mobile phones. AI is also making an impact on the entertainment industry. AI is capable of generating text, imagery, and possibly video in the future. Entertainment content delivery, user engagement, and augmented reality can all be improved with AI technologies [23][24].

Defense
As AI technology progresses, it opens up applications to the defense industry. Semi-autonomous UAVs may be developed using some technologies similar to those for autonomous driving. Many of the computer vision technologies can be adapted for automatic target detection, recognition, and tracking. Deep learning is applicable to the full spectrum of electronic warfare sensing. Most of technologies developed for civil use of robotics may be applicable to the defense domain for transportation of military supplies and other tasks. Regardless of the technology readiness level, scientists, engineers, media, and politicians should all be very careful when promoting AI for autonomous weapons [25]. Many ethics, legal, regulatory, and policy issues are not yet resolved.

Automatic Deep Machine Learning for mobile devices
As mentioned in previous sections, a deep learning for mobile device application has to be optimized for SWAP. There are many hierarchical parameters in the deep learning system that can be optimized. These are called hyper-parameter. Optimization of hyper-parameter is an active topic called automatic machine learning [51]. Some technology of automatic ML may be applied to mobile applications. The object function for mobile DL is more than just accuracy and can be a weighted as a function of accuracy, memory footprint, computing speed, and power consumption.

Low quality data and ML for signal processing
Mobile application often means the application is less constrained. Unlike the well-controlled data collected in the laboratory, mobile data is often noisier with more confounding factors. This could also mean some, if not a significant portion, of the labelled data can be wrong. Although DNN is robust to noisy label when the training data is large, it can be a problem with less training data [52]. The mobile deep learning engineer thus needs to be more careful to prepare the data for model training. Sometimes custom signal processing may be necessary to deal with some special phenomenon.

Small training data, few shot learning, and transfer learning
Mobile applications may often have less training data to start with, although unlabeled data may be more abundant. To kick start the system, few shot learning or transfer learning may be applied [48] [49]. As more data becomes available, the initial model can be improved iteratively.

Evolving environment, online adaptation, and lifelong learning
Another more challenging problem for mobile deep learning is that the environment can change over time. In some extreme cases, the system may have to face some unforeseen conditions. These situations require the mobile system to be able to perform online adaption to the change of the data or capable of lifelong learning [50].

Privacy in Mobile Deep Learning
As some mobile deep learning directly works on user personal data, privacy is a concern. This is particularly true if the data are stored and processed in the cloud. A hybrid deep learning architecture is recently proposed for privacypreserving mobile analytics [26].

ACKNOWLEDGEMENT
The author would like to thank Dr. Stephen DelMarco for the invitation to submit and many constructive comments on this paper. The author would also like to thank Mrs. Jennifer Krischer for reviewing this paper.