MAVIPER: Learning Decision Tree Policies for Interpretable Multi-Agent
  Reinforcement Learning
  Many recent breakthroughs in multi-agent reinforcement learning (MARL)
require the use of deep neural networks, which are challenging for human
experts to interpret and understand. On the other hand, existing work on
interpretable RL has shown promise in extracting more interpretable decision
tree-based policies, but only in the single-agent setting. To fill this gap, we
propose the first set of interpretable MARL algorithms that extract
decision-tree policies from neural networks trained with MARL. The first
algorithm, IVIPER, extends VIPER, a recent method for single-agent
interpretable RL, to the multi-agent setting. We demonstrate that IVIPER can
learn high-quality decision-tree policies for each agent. To better capture
coordination between agents, we propose a novel centralized decision-tree
training algorithm, MAVIPER. MAVIPER jointly grows the trees of each agent by
predicting the behavior of the other agents using their anticipated trees, and
uses resampling to focus on states that are critical for its interactions with
other agents. We show that both algorithms generally outperform the baselines
and that MAVIPER-trained agents achieve better-coordinated performance than
IVIPER-trained agents on three different multi-agent particle-world
environments.
