Intelligent Matrix Exponentiation
  We present a novel machine learning architecture that uses the exponential of
a single input-dependent matrix as its only nonlinearity. The mathematical
simplicity of this architecture allows a detailed analysis of its behaviour,
providing robustness guarantees via Lipschitz bounds. Despite its simplicity, a
single matrix exponential layer already provides universal approximation
properties and can learn fundamental functions of the input, such as periodic
functions or multivariate polynomials. This architecture outperforms other
general-purpose architectures on benchmark problems, including CIFAR-10, using
substantially fewer parameters.
