Benchmarking Safe Deep Reinforcement Learning in Aquatic Navigation
  We propose a novel benchmark environment for Safe Reinforcement Learning
focusing on aquatic navigation. Aquatic navigation is an extremely challenging
task due to the non-stationary environment and the uncertainties of the robotic
platform, hence it is crucial to consider the safety aspect of the problem, by
analyzing the behavior of the trained network to avoid dangerous situations
(e.g., collisions). To this end, we consider a value-based and policy-gradient
Deep Reinforcement Learning (DRL) and we propose a crossover-based strategy
that combines gradient-based and gradient-free DRL to improve
sample-efficiency. Moreover, we propose a verification strategy based on
interval analysis that checks the behavior of the trained models over a set of
desired properties. Our results show that the crossover-based training
outperforms prior DRL approaches, while our verification allows us to quantify
the number of configurations that violate the behaviors that are described by
the properties. Crucially, this will serve as a benchmark for future research
in this domain of applications.
