Synthesizing Context-free Grammars from Recurrent Neural Networks
  (Extended Version)
  We present an algorithm for extracting a subclass of the context free
grammars (CFGs) from a trained recurrent neural network (RNN). We develop a new
framework, pattern rule sets (PRSs), which describe sequences of deterministic
finite automata (DFAs) that approximate a non-regular language. We present an
algorithm for recovering the PRS behind a sequence of such automata, and apply
it to the sequences of automata extracted from trained RNNs using the L*
algorithm. We then show how the PRS may converted into a CFG, enabling a
familiar and useful presentation of the learned language.
  Extracting the learned language of an RNN is important to facilitate
understanding of the RNN and to verify its correctness. Furthermore, the
extracted CFG can augment the RNN in classifying correct sentences, as the
RNN's predictive accuracy decreases when the recursion depth and distance
between matching delimiters of its input sequences increases.
