Universal Equivariant Multilayer Perceptrons
  Group invariant and equivariant Multilayer Perceptrons (MLP), also known as
Equivariant Networks, have achieved remarkable success in learning on a variety
of data structures, such as sequences, images, sets, and graphs. Using tools
from group theory, this paper proves the universality of a broad class of
equivariant MLPs with a single hidden layer. In particular, it is shown that
having a hidden layer on which the group acts regularly is sufficient for
universal equivariance (invariance). A corollary is unconditional universality
of equivariant MLPs for Abelian groups, such as CNNs with a single hidden
layer. A second corollary is the universality of equivariant MLPs with a
high-order hidden layer, where we give both group-agnostic bounds and means for
calculating group-specific bounds on the order of hidden layer that guarantees
universal equivariance (invariance).
