References & Citations
Computer Science > Computation and Language
Title: Neural Embeddings for Text
(Submitted on 17 Aug 2022 (v1), last revised 20 Nov 2022 (this version, v2))
Abstract: We propose a new kind of embedding for natural language text that deeply represents semantic meaning. Standard text embeddings use the outputs from hidden layers of a pretrained language model. In our method, we let a language model learn from the text and then literally pick its brain, taking the actual weights of the model's neurons to generate a vector. We call this representation of the text a neural embedding. We confirm the ability of this representation to reflect semantics of the text by an analysis of its behavior on several datasets, and by a comparison of neural embedding with state of the art sentence embeddings.
Submission history
From: Oleg Vasilyev [view email][v1] Wed, 17 Aug 2022 16:26:13 GMT (60kb,D)
[v2] Sun, 20 Nov 2022 05:30:16 GMT (260kb,D)
Link back to: arXiv, form interface, contact.