Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement
  Learning Agents
  Robustness of Deep Reinforcement Learning (DRL) algorithms towards
adversarial attacks in real world applications such as those deployed in
cyber-physical systems (CPS) are of increasing concern. Numerous studies have
investigated the mechanisms of attacks on the RL agent's state space.
Nonetheless, attacks on the RL agent's action space (AS) (corresponding to
actuators in engineering systems) are equally perverse; such attacks are
relatively less studied in the ML literature. In this work, we first frame the
problem as an optimization problem of minimizing the cumulative reward of an RL
agent with decoupled constraints as the budget of attack. We propose a
white-box Myopic Action Space (MAS) attack algorithm that distributes the
attacks across the action space dimensions. Next, we reformulate the
optimization problem above with the same objective function, but with a
temporally coupled constraint on the attack budget to take into account the
approximated dynamics of the agent. This leads to the white-box Look-ahead
Action Space (LAS) attack algorithm that distributes the attacks across the
action and temporal dimensions. Our results shows that using the same amount of
resources, the LAS attack deteriorates the agent's performance significantly
more than the MAS attack. This reveals the possibility that with limited
resource, an adversary can utilize the agent's dynamics to malevolently craft
attacks that causes the agent to fail. Additionally, we leverage these attack
strategies as a possible tool to gain insights on the potential vulnerabilities
of DRL agents.
