An Empirical Evaluation of Rule Extraction from Recurrent Neural
  Networks
  Rule extraction from black-box models is critical in domains that require
model validation before implementation, as can be the case in credit scoring
and medical diagnosis. Though already a challenging problem in statistical
learning in general, the difficulty is even greater when highly non-linear,
recursive models, such as recurrent neural networks (RNNs), are fit to data.
Here, we study the extraction of rules from second-order recurrent neural
networks trained to recognize the Tomita grammars. We show that production
rules can be stably extracted from trained RNNs and that in certain cases the
rules outperform the trained RNNs.
