Scaffolding Reflection in Reinforcement Learning Framework for
  Confinement Escape Problem
  In this paper, a novel Scaffolding Reflection in Reinforcement Learning
(SR2L) is proposed for solving the confinement escape problem (CEP). In CEP, an
evader's objective is to attempt escaping a confinement region patrolled by
multiple pursuers. Meanwhile, the pursuers aim to reach and capture the evader.
The inverse solution for pursuers to try and capture has been extensively
studied in the literature. However, the problem of evaders escaping from the
region is still an open issue. The SR2L employs an actor-critic framework to
enable the evader to escape the confinement region. A time-varying state
representation and reward function have been developed for proper convergence.
The formulation uses the sensor information about the observable environment
and prior knowledge of the confinement boundary. The conventional Independent
Actor-Critic (IAC) method fails to converge due to sparseness in the reward.
The effect becomes evident when operating in such a dynamic environment with a
large area. In SR2L, along with the developed reward function, we use the
scaffolding reflection method to improve the convergence significantly while
increasing its efficiency. In SR2L, a motion planner is used as a scaffold for
the actor-critic network to observe, compare and learn the action-reward pair.
It enables the evader to achieve the required objective while using lesser
resources and time. Convergence studies show that SR2L learns faster and
converges to higher rewards as compared to IAC. Extensive Monte-Carlo
simulations show that a SR2L consistently outperforms conventional IAC and the
motion planner itself as the baselines.
