Deep Polynomial Neural Networks
  Deep Convolutional Neural Networks (DCNNs) are currently the method of choice
both for generative, as well as for discriminative learning in computer vision
and machine learning. The success of DCNNs can be attributed to the careful
selection of their building blocks (e.g., residual blocks, rectifiers,
sophisticated normalization schemes, to mention but a few). In this paper, we
propose $\Pi$-Nets, a new class of function approximators based on polynomial
expansions. $\Pi$-Nets are polynomial neural networks, i.e., the output is a
high-order polynomial of the input. The unknown parameters, which are naturally
represented by high-order tensors, are estimated through a collective tensor
factorization with factors sharing. We introduce three tensor decompositions
that significantly reduce the number of parameters and show how they can be
efficiently implemented by hierarchical neural networks. We empirically
demonstrate that $\Pi$-Nets are very expressive and they even produce good
results without the use of non-linear activation functions in a large battery
of tasks and signals, i.e., images, graphs, and audio. When used in conjunction
with activation functions, $\Pi$-Nets produce state-of-the-art results in three
challenging tasks, i.e. image generation, face verification and 3D mesh
representation learning. The source code is available at
\url{https://github.com/grigorisg9gr/polynomial_nets}.
