Risky Action Recognition in Lane Change Video Clips using Deep
  Spatiotemporal Networks with Segmentation Mask Transfer
  Advanced driver assistance and automated driving systems rely on risk
estimation modules to predict and avoid dangerous situations. Current methods
use expensive sensor setups and complex processing pipeline, limiting their
availability and robustness. To address these issues, we introduce a novel deep
learning based action recognition framework for classifying dangerous lane
change behavior in short video clips captured by a monocular camera. We
designed a deep spatiotemporal classification network that uses pre-trained
state-of-the-art instance segmentation network Mask R-CNN as its spatial
feature extractor for this task. The Long-Short Term Memory (LSTM) and
shallower final classification layers of the proposed method were trained on a
semi-naturalistic lane change dataset with annotated risk labels. A
comprehensive comparison of state-of-the-art feature extractors was carried out
to find the best network layout and training strategy. The best result, with a
0.937 AUC score, was obtained with the proposed network. Our code and trained
models are available open-source.
