DFTerNet: Towards 2-bit Dynamic Fusion Networks for Accurate Human
  Activity Recognition
  Deep Convolutional Neural Networks (DCNNs) are currently popular in human
activity recognition applications. However, in the face of modern artificial
intelligence sensor-based games, many research achievements cannot be
practically applied on portable devices. DCNNs are typically resource-intensive
and too large to be deployed on portable devices, thus this limits the
practical application of complex activity detection. In addition, since
portable devices do not possess high-performance Graphic Processing Units
(GPUs), there is hardly any improvement in Action Game (ACT) experience.
Besides, in order to deal with multi-sensor collaboration, all previous human
activity recognition models typically treated the representations from
different sensor signal sources equally. However, distinct types of activities
should adopt different fusion strategies. In this paper, a novel scheme is
proposed. This scheme is used to train 2-bit Convolutional Neural Networks with
weights and activations constrained to {-0.5,0,0.5}. It takes into account the
correlation between different sensor signal sources and the activity types.
This model, which we refer to as DFTerNet, aims at producing a more reliable
inference and better trade-offs for practical applications. Our basic idea is
to exploit quantization of weights and activations directly in pre-trained
filter banks and adopt dynamic fusion strategies for different activity types.
Experiments demonstrate that by using dynamic fusion strategy can exceed the
baseline model performance by up to ~5% on activity recognition like
OPPORTUNITY and PAMAP2 datasets. Using the quantization method proposed, we
were able to achieve performances closer to that of full-precision counterpart.
These results were also verified using the UniMiB-SHAR dataset. In addition,
the proposed method can achieve ~9x acceleration on CPUs and ~11x memory
saving.
