A Deep Learning-based Multimodal Depth-Aware Dynamic Hand Gesture
  Recognition System
  The dynamic hand gesture recognition task has seen studies on various
unimodal and multimodal methods. Previously, researchers have explored depth
and 2D-skeleton-based multimodal fusion CRNNs (Convolutional Recurrent Neural
Networks) but have had limitations in getting expected recognition results. In
this paper, we revisit this approach to hand gesture recognition and suggest
several improvements. We observe that raw depth images possess low contrast in
the hand regions of interest (ROI). They do not highlight important fine
details, such as finger orientation, overlap between the finger and palm, or
overlap between multiple fingers. We thus propose quantizing the depth values
into several discrete regions, to create a higher contrast between several key
parts of the hand. In addition, we suggest several ways to tackle the high
variance problem in existing multimodal fusion CRNN architectures. We evaluate
our method on two benchmarks: the DHG-14/28 dataset and the SHREC'17 track
dataset. Our approach shows a significant improvement in accuracy and parameter
efficiency over previous similar multimodal methods, with a comparable result
to the state-of-the-art.
