Current browse context:
cs.CL
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title: Squashed Shifted PMI Matrix: Bridging Word Embeddings and Hyperbolic Spaces
(Submitted on 27 Feb 2020 (v1), last revised 26 Sep 2020 (this version, v2))
Abstract: We show that removing sigmoid transformation in the skip-gram with negative sampling (SGNS) objective does not harm the quality of word vectors significantly and at the same time is related to factorizing a squashed shifted PMI matrix which, in turn, can be treated as a connection probabilities matrix of a random graph. Empirically, such graph is a complex network, i.e. it has strong clustering and scale-free degree distribution, and is tightly connected with hyperbolic spaces. In short, we show the connection between static word embeddings and hyperbolic spaces through the squashed shifted PMI matrix using analytical and empirical methods.
Submission history
From: Zhenisbek Assylbekov [view email][v1] Thu, 27 Feb 2020 09:50:41 GMT (1251kb,D)
[v2] Sat, 26 Sep 2020 15:06:00 GMT (651kb,D)
Link back to: arXiv, form interface, contact.