ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via
  Convex Relaxation
  In a multirobot system, a number of cyber-physical attacks (e.g.,
communication hijack, observation perturbations) can challenge the robustness
of agents. This robustness issue worsens in multiagent reinforcement learning
because there exists the non-stationarity of the environment caused by
simultaneously learning agents whose changing policies affect the transition
and reward functions. In this paper, we propose a minimax MARL approach to
infer the worst-case policy update of other agents. As the minimax formulation
is computationally intractable to solve, we apply the convex relaxation of
neural networks to solve the inner minimization problem. Such convex relaxation
enables robustness in interacting with peer agents that may have significantly
different behaviors and also achieves a certified bound of the original
optimization problem. We evaluate our approach on multiple mixed
cooperative-competitive tasks and show that our method outperforms the previous
state of the art approaches on this topic.
