Fullie and Wiselie: A Dual-Stream Recurrent Convolutional Attention
  Model for Activity Recognition
  Multimodal features play a key role in wearable sensor based Human Activity
Recognition (HAR). Selecting the most salient features adaptively is a
promising way to maximize the effectiveness of multimodal sensor data. In this
regard, we propose a "collect fully and select wisely (Fullie and Wiselie)"
principle as well as a dual-stream recurrent convolutional attention model,
Recurrent Attention and Activity Frame (RAAF), to improve the recognition
performance. We first collect modality features and the relations between each
pair of features to generate activity frames, and then introduce an attention
mechanism to select the most prominent regions from activity frames precisely.
The selected frames not only maximize the utilization of valid features but
also reduce the number of features to be computed effectively. We further
analyze the hyper-parameters, accuracy, interpretability, and annotation
dependency of the proposed model based on extensive experiments. The results
show that RAAF achieves competitive performance on two benchmarked datasets and
works well in real life scenarios.
