Optimization Methods for Interpretable Differentiable Decision Trees in
  Reinforcement Learning
  Decision trees are ubiquitous in machine learning for their ease of use and
interpretability. Yet, these models are not typically employed in reinforcement
learning as they cannot be updated online via stochastic gradient descent. We
overcome this limitation by allowing for a gradient update over the entire tree
that improves sample complexity affords interpretable policy extraction. First,
we include theoretical motivation on the need for policy-gradient learning by
examining the properties of gradient descent over differentiable decision
trees. Second, we demonstrate that our approach equals or outperforms a neural
network on all domains and can learn discrete decision trees online with
average rewards up to 7x higher than a batch-trained decision tree. Third, we
conduct a user study to quantify the interpretability of a decision tree, rule
list, and a neural network with statistically significant results ($p <
0.001$).
