Recurrent Graph Tensor Networks: A Low-Complexity Framework for
  Modelling High-Dimensional Multi-Way Sequence
  Recurrent Neural Networks (RNNs) are among the most successful machine
learning models for sequence modelling, but tend to suffer from an exponential
increase in the number of parameters when dealing with large multidimensional
data. To this end, we develop a multi-linear graph filter framework for
approximating the modelling of hidden states in RNNs, which is embedded in a
tensor network architecture to improve modelling power and reduce parameter
complexity, resulting in a novel Recurrent Graph Tensor Network (RGTN). The
proposed framework is validated through several multi-way sequence modelling
tasks and benchmarked against traditional RNNs. By virtue of the domain aware
information processing of graph filters and the expressive power of tensor
networks, we show that the proposed RGTN is capable of not only out-performing
standard RNNs, but also mitigating the Curse of Dimensionality associated with
traditional RNNs, demonstrating superior properties in terms of performance and
complexity.
