Nested Hyperbolic Spaces for Dimensionality Reduction and Hyperbolic NN
  Design
  Hyperbolic neural networks have been popular in the recent past due to their
ability to represent hierarchical data sets effectively and efficiently. The
challenge in developing these networks lies in the nonlinearity of the
embedding space namely, the Hyperbolic space. Hyperbolic space is a homogeneous
Riemannian manifold of the Lorentz group. Most existing methods (with some
exceptions) use local linearization to define a variety of operations
paralleling those used in traditional deep neural networks in Euclidean spaces.
In this paper, we present a novel fully hyperbolic neural network which uses
the concept of projections (embeddings) followed by an intrinsic aggregation
and a nonlinearity all within the hyperbolic space. The novelty here lies in
the projection which is designed to project data on to a lower-dimensional
embedded hyperbolic space and hence leads to a nested hyperbolic space
representation independently useful for dimensionality reduction. The main
theoretical contribution is that the proposed embedding is proved to be
isometric and equivariant under the Lorentz transformations. This projection is
computationally efficient since it can be expressed by simple linear
operations, and, due to the aforementioned equivariance property, it allows for
weight sharing. The nested hyperbolic space representation is the core
component of our network and therefore, we first compare this ensuing nested
hyperbolic space representation with other dimensionality reduction methods
such as tangent PCA, principal geodesic analysis (PGA) and HoroPCA. Based on
this equivariant embedding, we develop a novel fully hyperbolic graph
convolutional neural network architecture to learn the parameters of the
projection. Finally, we present experiments demonstrating comparative
performance of our network on several publicly available data sets.
