Improved Generalization Bounds of Group Invariant / Equivariant Deep
  Networks via Quotient Feature Spaces
  Numerous invariant (or equivariant) neural networks have succeeded in
handling invariant data such as point clouds and graphs. However, a
generalization theory for the neural networks has not been well developed,
because several essential factors for the theory, such as network size and
margin distribution, are not deeply connected to the invariance and
equivariance. In this study, we develop a novel generalization error bound for
invariant and equivariant deep neural networks. To describe the effect of
invariance and equivariance on generalization, we develop a notion of a
\textit{quotient feature space}, which measures the effect of group actions for
the properties. Our main result proves that the volume of quotient feature
spaces can describe the generalization error. Furthermore, the bound shows that
the invariance and equivariance significantly improve the leading term of the
bound. We apply our result to specific invariant and equivariant networks, such
as DeepSets (Zaheer et al. (2017)), and show that their generalization bound is
considerably improved by $\sqrt{n!}$, where $n!$ is the number of permutations.
We also discuss the expressive power of invariant DNNs and show that they can
achieve an optimal approximation rate. Our experimental result supports our
theoretical claims.
