DeepSense: A Unified Deep Learning Framework for Time-Series Mobile
  Sensing Data Processing
  Mobile sensing applications usually require time-series inputs from sensors.
Some applications, such as tracking, can use sensed acceleration and rate of
rotation to calculate displacement based on physical system models. Other
applications, such as activity recognition, extract manually designed features
from sensor inputs for classification. Such applications face two challenges.
On one hand, on-device sensor measurements are noisy. For many mobile
applications, it is hard to find a distribution that exactly describes the
noise in practice. Unfortunately, calculating target quantities based on
physical system and noise models is only as accurate as the noise assumptions.
Similarly, in classification applications, although manually designed features
have proven to be effective, it is not always straightforward to find the most
robust features to accommodate diverse sensor noise patterns and user
behaviors. To this end, we propose DeepSense, a deep learning framework that
directly addresses the aforementioned noise and feature customization
challenges in a unified manner. DeepSense integrates convolutional and
recurrent neural networks to exploit local interactions among similar mobile
sensors, merge local interactions of different sensory modalities into global
interactions, and extract temporal relationships to model signal dynamics.
DeepSense thus provides a general signal estimation and classification
framework that accommodates a wide range of applications. We demonstrate the
effectiveness of DeepSense using three representative and challenging tasks:
car tracking with motion sensors, heterogeneous human activity recognition, and
user identification with biometric motion analysis. DeepSense significantly
outperforms the state-of-the-art methods for all three tasks. In addition,
DeepSense is feasible to implement on smartphones due to its moderate energy
consumption and low latency
