Geometric Deep Learning and Equivariant Neural Networks
  We survey the mathematical foundations of geometric deep learning, focusing
on group equivariant and gauge equivariant neural networks. We develop gauge
equivariant convolutional neural networks on arbitrary manifolds $\mathcal{M}$
using principal bundles with structure group $K$ and equivariant maps between
sections of associated vector bundles. We also discuss group equivariant neural
networks for homogeneous spaces $\mathcal{M}=G/K$, which are instead
equivariant with respect to the global symmetry $G$ on $\mathcal{M}$. Group
equivariant layers can be interpreted as intertwiners between induced
representations of $G$, and we show their relation to gauge equivariant
convolutional layers. We analyze several applications of this formalism,
including semantic segmentation and object detection networks. We also discuss
the case of spherical networks in great detail, corresponding to the case
$\mathcal{M}=S^2=\mathrm{SO}(3)/\mathrm{SO}(2)$. Here we emphasize the use of
Fourier analysis involving Wigner matrices, spherical harmonics and
Clebsch-Gordan coefficients for $G=\mathrm{SO}(3)$, illustrating the power of
representation theory for deep learning.
