Overlapping Spaces for Compact Graph Representations
  Various non-trivial spaces are becoming popular for embedding structured data
such as graphs, texts, or images. Following spherical and hyperbolic spaces,
more general product spaces have been proposed. However, searching for the best
configuration of product space is a resource-intensive procedure, which reduces
the practical applicability of the idea. We generalize the concept of product
space and introduce an overlapping space that does not have the configuration
search problem. The main idea is to allow subsets of coordinates to be shared
between spaces of different types (Euclidean, hyperbolic, spherical). As a
result, parameter optimization automatically learns the optimal configuration.
Additionally, overlapping spaces allow for more compact representations since
their geometry is more complex. Our experiments confirm that overlapping spaces
outperform the competitors in graph embedding tasks. Here, we consider both
distortion setup, where the aim is to preserve distances, and ranking setup,
where the relative order should be preserved. The proposed method effectively
solves the problem and outperforms the competitors in both settings. We also
perform an empirical analysis in a realistic information retrieval task, where
we compare all spaces by incorporating them into DSSM. In this case, the
proposed overlapping space consistently achieves nearly optimal results without
any configuration tuning. This allows for reducing training time, which can be
significant in large-scale applications.
