Improving Local Identifiability in Probabilistic Box Embeddings
  Geometric embeddings have recently received attention for their natural
ability to represent transitive asymmetric relations via containment. Box
embeddings, where objects are represented by n-dimensional hyperrectangles, are
a particularly promising example of such an embedding as they are closed under
intersection and their volume can be calculated easily, allowing them to
naturally represent calibrated probability distributions. The benefits of
geometric embeddings also introduce a problem of local identifiability,
however, where whole neighborhoods of parameters result in equivalent loss
which impedes learning. Prior work addressed some of these issues by using an
approximation to Gaussian convolution over the box parameters, however, this
intersection operation also increases the sparsity of the gradient. In this
work, we model the box parameters with min and max Gumbel distributions, which
were chosen such that space is still closed under the operation of the
intersection. The calculation of the expected intersection volume involves all
parameters, and we demonstrate experimentally that this drastically improves
the ability of such models to learn.
