Backdoor Detection in Reinforcement Learning
  While the real world application of reinforcement learning (RL) is becoming
popular, the safety concern and the robustness of an RL system require more
attention. A recent work reveals that, in a multi-agent RL environment,
backdoor trigger actions can be injected into a victim agent (a.k.a. trojan
agent), which can result in a catastrophic failure as soon as it sees the
backdoor trigger action. We propose the problem of RL Backdoor Detection,
aiming to address this safety vulnerability. An interesting observation we drew
from extensive empirical studies is a trigger smoothness property where normal
actions similar to the backdoor trigger actions can also trigger low
performance of the trojan agent. Inspired by this observation, we propose a
reinforcement learning solution TrojanSeeker to find approximate trigger
actions for the trojan agents, and further propose an efficient approach to
mitigate the trojan agents based on machine unlearning. Experiments show that
our approach can correctly distinguish and mitigate all the trojan agents
across various types of agents and environments.
