We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling

Abstract: Incorporating lexical knowledge into deep learning models has been proved to be very effective for sequence labeling tasks. However, previous works commonly have difficulty dealing with large-scale dynamic lexicons which often cause excessive matching noise and problems of frequent updates. In this paper, we propose DyLex, a plug-in lexicon incorporation approach for BERT based sequence labeling tasks. Instead of leveraging embeddings of words in the lexicon as in conventional methods, we adopt word-agnostic tag embeddings to avoid re-training the representation while updating the lexicon. Moreover, we employ an effective supervised lexical knowledge denoising method to smooth out matching noise. Finally, we introduce a col-wise attention based knowledge fusion mechanism to guarantee the pluggability of the proposed framework. Experiments on ten datasets of three tasks show that the proposed framework achieves new SOTA, even with very large scale lexicons.
Comments: EMNLP 2021 Long Paper
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2109.08818 [cs.CL]
  (or arXiv:2109.08818v2 [cs.CL] for this version)

Submission history

From: Baojun Wang [view email]
[v1] Sat, 18 Sep 2021 03:15:49 GMT (1380kb,D)
[v2] Wed, 22 Sep 2021 04:19:04 GMT (1238kb,D)

Link back to: arXiv, form interface, contact.