LE-HGR: A Lightweight and Efficient RGB-based Online Gesture Recognition
  Network for Embedded AR Devices
  Online hand gesture recognition (HGR) techniques are essential in augmented
reality (AR) applications for enabling natural human-to-computer interaction
and communication. In recent years, the consumer market for low-cost AR devices
has been rapidly growing, while the technology maturity in this domain is still
limited. Those devices are typical of low prices, limited memory, and
resource-constrained computational units, which makes online HGR a challenging
problem. To tackle this problem, we propose a lightweight and computationally
efficient HGR framework, namely LE-HGR, to enable real-time gesture recognition
on embedded devices with low computing power. We also show that the proposed
method is of high accuracy and robustness, which is able to reach high-end
performance in a variety of complicated interaction environments. To achieve
our goal, we first propose a cascaded multi-task convolutional neural network
(CNN) to simultaneously predict probabilities of hand detection and regress
hand keypoint locations online. We show that, with the proposed cascaded
architecture design, false-positive estimates can be largely eliminated.
Additionally, an associated mapping approach is introduced to track the hand
trace via the predicted locations, which addresses the interference of
multi-handedness. Subsequently, we propose a trace sequence neural network
(TraceSeqNN) to recognize the hand gesture by exploiting the motion features of
the tracked trace. Finally, we provide a variety of experimental results to
show that the proposed framework is able to achieve state-of-the-art accuracy
with significantly reduced computational cost, which are the key properties for
enabling real-time applications in low-cost commercial devices such as mobile
devices and AR/VR headsets.
