Beyond Value: CHECKLIST for Testing Inferences in Planning-Based RL
  Reinforcement learning (RL) agents are commonly evaluated via their expected
value over a distribution of test scenarios. Unfortunately, this evaluation
approach provides limited evidence for post-deployment generalization beyond
the test distribution. In this paper, we address this limitation by extending
the recent CheckList testing methodology from natural language processing to
planning-based RL. Specifically, we consider testing RL agents that make
decisions via online tree search using a learned transition model and value
function. The key idea is to improve the assessment of future performance via a
CheckList approach for exploring and assessing the agent's inferences during
tree search. The approach provides the user with an interface and general
query-rule mechanism for identifying potential inference flaws and validating
expected inference invariances. We present a user study involving knowledgeable
AI researchers using the approach to evaluate an agent trained to play a
complex real-time strategy game. The results show the approach is effective in
allowing users to identify previously-unknown flaws in the agent's reasoning.
In addition, our analysis provides insight into how AI experts use this type of
testing approach, which may help improve future instantiations.
