Infinite-Horizon Reach-Avoid Zero-Sum Games via Deep Reinforcement
  Learning
  In this paper, we consider the infinite-horizon reach-avoid zero-sum game
problem, where the goal is to find a set in the state space, referred to as the
reach-avoid set, such that the system starting at a state therein could be
controlled to reach a given target set without violating constraints under the
worst-case disturbance. We address this problem by designing a new value
function with a contracting Bellman backup, where the super-zero level set,
i.e., the set of states where the value function is evaluated to be
non-negative, recovers the reach-avoid set. Building upon this, we prove that
the proposed method can be adapted to compute the viability kernel, or the set
of states which could be controlled to satisfy given constraints, and the
backward reachable set, or the set of states that could be driven towards a
given target set. Finally, we propose to alleviate the curse of dimensionality
issue in high-dimensional problems by extending Conservative Q-Learning, a deep
reinforcement learning technique, to learn a value function such that the
super-zero level set of the learned value function serves as a (conservative)
approximation to the reach-avoid set. Our theoretical and empirical results
suggest that the proposed method could learn reliably the reach-avoid set and
the optimal control policy even with neural network approximation.
