Transformationally Identical and Invariant Convolutional Neural Networks
  by Combining Symmetric Operations or Input Vectors
  Transformationally invariant processors constructed by transformed input
vectors or operators have been suggested and applied to many applications. In
this study, transformationally identical processing based on combining results
of all sub-processes with corresponding transformations at one of the
processing steps or at the beginning step were found to be equivalent for a
given condition. This property can be applied to most convolutional neural
network (CNN) systems. Specifically, a transformationally identical CNN can be
constructed by arranging internally symmetric operations in parallel with the
same transformation family that includes a flatten layer with weights sharing
among their corresponding transformation elements. Other transformationally
identical CNNs can be constructed by averaging transformed input vectors of the
family at the input layer followed by an ordinary CNN process or by a set of
symmetric operations. Interestingly, we found that both types of
transformationally identical CNN systems are mathematically equivalent by
either applying an averaging operation to corresponding elements of all
sub-channels before the activation function or without using a non-linear
activation function.
