Neural Network Based Nonlinear Weighted Finite Automata
  Weighted finite automata (WFA) can expressively model functions defined over
strings but are inherently linear models. Given the recent successes of
nonlinear models in machine learning, it is natural to wonder whether
ex-tending WFA to the nonlinear setting would be beneficial. In this paper, we
propose a novel model of neural network based nonlinearWFA model (NL-WFA) along
with a learning algorithm. Our learning algorithm is inspired by the spectral
learning algorithm for WFAand relies on a nonlinear decomposition of the
so-called Hankel matrix, by means of an auto-encoder network. The expressive
power of NL-WFA and the proposed learning algorithm are assessed on both
synthetic and real-world data, showing that NL-WFA can lead to smaller model
sizes and infer complex grammatical structures from data.
