We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Kernelized Bayesian Softmax for Text Generation

Abstract: Neural models for text generation require a softmax layer with proper token embeddings during the decoding phase. Most existing approaches adopt single point embedding for each token. However, a word may have multiple senses according to different context, some of which might be distinct. In this paper, we propose KerBS, a novel approach for learning better embeddings for text generation. KerBS embodies two advantages: (a) it employs a Bayesian composition of embeddings for words with multiple senses; (b) it is adaptive to semantic variances of words and robust to rare sentence context by imposing learned kernels to capture the closeness of words (senses) in the embedding space. Empirical studies show that KerBS significantly boosts the performance of several text generation tasks.
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:1911.00274 [cs.CL]
  (or arXiv:1911.00274v1 [cs.CL] for this version)

Submission history

From: Ning Miao [view email]
[v1] Fri, 1 Nov 2019 09:20:14 GMT (631kb,D)

Link back to: arXiv, form interface, contact.