Generalising Recursive Neural Models by Tensor Decomposition
  Most machine learning models for structured data encode the structural
knowledge of a node by leveraging simple aggregation functions (in neural
models, typically a weighted sum) of the information in the node's
neighbourhood. Nevertheless, the choice of simple context aggregation
functions, such as the sum, can be widely sub-optimal. In this work we
introduce a general approach to model aggregation of structural context
leveraging a tensor-based formulation. We show how the exponential growth in
the size of the parameter space can be controlled through an approximation
based on the Tucker tensor decomposition. This approximation allows limiting
the parameters space size, decoupling it from its strict relation with the size
of the hidden encoding space. By this means, we can effectively regulate the
trade-off between expressivity of the encoding, controlled by the hidden size,
computational complexity and model generalisation, influenced by
parameterisation. Finally, we introduce a new Tensorial Tree-LSTM derived as an
instance of our framework and we use it to experimentally assess our working
hypotheses on tree classification scenarios.
