ManifoldNorm: Extending normalizations on Riemannian Manifolds
  Many measurements in computer vision and machine learning manifest as
non-Euclidean data samples. Several researchers recently extended a number of
deep neural network architectures for manifold valued data samples. Researchers
have proposed models for manifold valued spatial data which are common in
medical image processing including processing of diffusion tensor imaging (DTI)
where images are fields of $3\times 3$ symmetric positive definite matrices or
representation in terms of orientation distribution field (ODF) where the
identification is in terms of field on hypersphere. There are other sequential
models for manifold valued data that recently researchers have shown to be
effective for group difference analysis in study for neuro-degenerative
diseases. Although, several of these methods are effective to deal with
manifold valued data, the bottleneck includes the instability in optimization
for deeper networks. In order to deal with these instabilities, researchers
have proposed residual connections for manifold valued data. One of the other
remedies to deal with the instabilities including gradient explosion is to use
normalization techniques including {\it batch norm} and {\it group norm} etc..
But, so far there is no normalization techniques applicable for manifold valued
data. In this work, we propose a general normalization techniques for manifold
valued data. We show that our proposed manifold normalization technique have
special cases including popular batch norm and group norm techniques. On the
experimental side, we focus on two types of manifold valued data including
manifold of symmetric positive definite matrices and hypersphere. We show the
performance gain in one synthetic experiment for moving MNIST dataset and one
real brain image dataset where the representation is in terms of orientation
distribution field (ODF).
