Dynamic Gesture Recognition by Using CNNs and Star RGB: a Temporal
  Information Condensation
  Due to the advance of technologies, machines are increasingly present in
people's daily lives. Thus, there has been more and more effort to develop
interfaces, such as dynamic gestures, that provide an intuitive way of
interaction. Currently, the most common trend is to use multimodal data, as
depth and skeleton information, to enable dynamic gesture recognition. However,
using only color information would be more interesting, since RGB cameras are
usually available in almost every public place, and could be used for gesture
recognition without the need of installing other equipment. The main problem
with such approach is the difficulty of representing spatio-temporal
information using just color. With this in mind, we propose a technique capable
of condensing a dynamic gesture, shown in a video, in just one RGB image. We
call this technique star RGB. This image is then passed to a classifier formed
by two Resnet CNNs, a soft-attention ensemble, and a fully connected layer,
which indicates the class of the gesture present in the input video.
Experiments were carried out using both Montalbano and GRIT datasets. For
Montalbano dataset, the proposed approach achieved an accuracy of 94.58%. Such
result reaches the state-of-the-art when considering this dataset and only
color information. Regarding the GRIT dataset, our proposal achieves more than
98% of accuracy, recall, precision, and F1-score, outperforming the reference
approach by more than 6%.
