References & Citations
Computer Science > Computation and Language
Title: Improving Interpretability of Word Embeddings by Generating Definition and Usage
(Submitted on 12 Dec 2019 (v1), last revised 18 Jul 2020 (this version, v2))
Abstract: Word embeddings are substantially successful in capturing semantic relations among words. However, these lexical semantics are difficult to be interpreted. Definition modeling provides a more intuitive way to evaluate embeddings by utilizing them to generate natural language definitions of corresponding words. This task is of great significance for practical application and in-depth understanding of word representations. We propose a novel framework for definition modeling, which can generate reasonable and understandable context-dependent definitions. Moreover, we introduce usage modeling and study whether it is possible to utilize embeddings to generate example sentences of words. These ways are a more direct and explicit expression of embedding's semantics for better interpretability. We extend the single task model to multi-task setting and investigate several joint multi-task models to combine usage modeling and definition modeling together. Experimental results on existing Oxford dataset and a new collected Oxford-2019 dataset show that our single-task model achieves the state-of-the-art result in definition modeling and the multi-task learning methods are helpful for two tasks to improve the performance.
Submission history
From: Haitong Zhang [view email][v1] Thu, 12 Dec 2019 12:45:34 GMT (388kb,D)
[v2] Sat, 18 Jul 2020 15:17:38 GMT (733kb,D)
Link back to: arXiv, form interface, contact.