We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: SimCLAD: A Simple Framework for Contrastive Learning of Acronym Disambiguation

Abstract: Acronym disambiguation means finding the correct meaning of an ambiguous acronym from the dictionary in a given sentence, which is one of the key points for scientific document understanding (SDU@AAAI-22). Recently, many attempts have tried to solve this problem via fine-tuning the pre-trained masked language models (MLMs) in order to obtain a better acronym representation. However, the acronym meaning is varied under different contexts, whose corresponding phrase representation mapped in different directions lacks discrimination in the entire vector space. Thus, the original representations of the pre-trained MLMs are not ideal for the acronym disambiguation task. In this paper, we propose a Simple framework for Contrastive Learning of Acronym Disambiguation (SimCLAD) method to better understand the acronym meanings. Specifically, we design a continual contrastive pre-training method that enhances the pre-trained model's generalization ability by learning the phrase-level contrastive distributions between true meaning and ambiguous phrases. The results on the acronym disambiguation of the scientific domain in English show that the proposed method outperforms all other competitive state-of-the-art (SOTA) methods.
Comments: Accepted for Artificial Intelligence on Scientific Document Understanding (SDU) workshop at AAAI 2022
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2111.14306 [cs.CL]
  (or arXiv:2111.14306v3 [cs.CL] for this version)

Submission history

From: Bin Li [view email]
[v1] Mon, 29 Nov 2021 02:39:59 GMT (1369kb,D)
[v2] Sun, 5 Dec 2021 12:17:42 GMT (1556kb,D)
[v3] Thu, 9 Dec 2021 09:43:42 GMT (1557kb,D)

Link back to: arXiv, form interface, contact.