Improving Variational Autoencoder based Out-of-Distribution Detection
  for Embedded Real-time Applications
  Uncertainties in machine learning are a significant roadblock for its
application in safety-critical cyber-physical systems (CPS). One source of
uncertainty arises from distribution shifts in the input data between training
and test scenarios. Detecting such distribution shifts in real-time is an
emerging approach to address the challenge. The high dimensional input space in
CPS applications involving imaging adds extra difficulty to the task.
Generative learning models are widely adopted for the task, namely
out-of-distribution (OoD) detection. To improve the state-of-the-art, we
studied existing proposals from both machine learning and CPS fields. In the
latter, safety monitoring in real-time for autonomous driving agents has been a
focus. Exploiting the spatiotemporal correlation of motion in videos, we can
robustly detect hazardous motion around autonomous driving agents. Inspired by
the latest advances in the Variational Autoencoder (VAE) theory and practice,
we tapped into the prior knowledge in data to further boost OoD detection's
robustness. Comparison studies over nuScenes and Synthia data sets show our
methods significantly improve detection capabilities of OoD factors unique to
driving scenarios, 42% better than state-of-the-art approaches. Our model also
generalized near-perfectly, 97% better than the state-of-the-art across the
real-world and simulation driving data sets experimented. Finally, we
customized one proposed method into a twin-encoder model that can be deployed
to resource limited embedded devices for real-time OoD detection. Its execution
time was reduced over four times in low-precision 8-bit integer inference,
while detection capability is comparable to its corresponding floating-point
model.
