Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Improving Local Identifiability in Probabilistic Box Embeddings
(Submitted on 9 Oct 2020 (v1), last revised 29 Oct 2020 (this version, v2))
Abstract: Geometric embeddings have recently received attention for their natural ability to represent transitive asymmetric relations via containment. Box embeddings, where objects are represented by n-dimensional hyperrectangles, are a particularly promising example of such an embedding as they are closed under intersection and their volume can be calculated easily, allowing them to naturally represent calibrated probability distributions. The benefits of geometric embeddings also introduce a problem of local identifiability, however, where whole neighborhoods of parameters result in equivalent loss which impedes learning. Prior work addressed some of these issues by using an approximation to Gaussian convolution over the box parameters, however, this intersection operation also increases the sparsity of the gradient. In this work, we model the box parameters with min and max Gumbel distributions, which were chosen such that space is still closed under the operation of the intersection. The calculation of the expected intersection volume involves all parameters, and we demonstrate experimentally that this drastically improves the ability of such models to learn.
Submission history
From: Shib Sankar Dasgupta [view email][v1] Fri, 9 Oct 2020 22:34:12 GMT (727kb,D)
[v2] Thu, 29 Oct 2020 01:39:49 GMT (1107kb,D)
Link back to: arXiv, form interface, contact.