ConformalLayers: A non-linear sequential neural network with associative
  layers
  Convolutional Neural Networks (CNNs) have been widely applied. But as the
CNNs grow, the number of arithmetic operations and memory footprint also
increase. Furthermore, typical non-linear activation functions do not allow
associativity of the operations encoded by consecutive layers, preventing the
simplification of intermediate steps by combining them. We present a new
activation function that allows associativity between sequential layers of
CNNs. Even though our activation function is non-linear, it can be represented
by a sequence of linear operations in the conformal model for Euclidean
geometry. In this domain, operations like, but not limited to, convolution,
average pooling, and dropout remain linear. We take advantage of associativity
to combine all the "conformal layers" and make the cost of inference constant
regardless of the depth of the network.
