Real-Time Driver State Monitoring Using a CNN Based Spatio-Temporal
  Approach
  Many road accidents occur due to distracted drivers. Today, driver monitoring
is essential even for the latest autonomous vehicles to alert distracted
drivers in order to take over control of the vehicle in case of emergency. In
this paper, a spatio-temporal approach is applied to classify drivers'
distraction level and movement decisions using convolutional neural networks
(CNNs). We approach this problem as action recognition to benefit from temporal
information in addition to spatial information. Our approach relies on features
extracted from sparsely selected frames of an action using a pre-trained
BN-Inception network. Experiments show that our approach outperforms the
state-of-the art results on the Distracted Driver Dataset (96.31%), with an
accuracy of 99.10% for 10-class classification while providing real-time
performance. We also analyzed the impact of fusion using RGB and optical flow
modalities with a very recent data level fusion strategy. The results on the
Distracted Driver and Brain4Cars datasets show that fusion of these modalities
further increases the accuracy.
