Benchmark for Out-of-Distribution Detection in Deep Reinforcement
  Learning
  Reinforcement Learning (RL) based solutions are being adopted in a variety of
domains including robotics, health care and industrial automation. Most focus
is given to when these solutions work well, but they fail when presented with
out of distribution inputs. RL policies share the same faults as most machine
learning models. Out of distribution detection for RL is generally not well
covered in the literature, and there is a lack of benchmarks for this task. In
this work we propose a benchmark to evaluate OOD detection methods in a
Reinforcement Learning setting, by modifying the physical parameters of
non-visual standard environments or corrupting the state observation for visual
environments. We discuss ways to generate custom RL environments that can
produce OOD data, and evaluate three uncertainty methods for the OOD detection
task. Our results show that ensemble methods have the best OOD detection
performance with a lower standard deviation across multiple environments.
