We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Enhancing Clinical Information Extraction with Transferred Contextual Embeddings

Abstract: The Bidirectional Encoder Representations from Transformers (BERT) model has achieved the state-of-the-art performance for many natural language processing (NLP) tasks. Yet, limited research has been contributed to studying its effectiveness when the target domain is shifted from the pre-training corpora, for example, for biomedical or clinical NLP applications. In this paper, we applied it to a widely studied a hospital information extraction (IE) task and analyzed its performance under the transfer learning setting. Our application became the new state-of-the-art result by a clear margin, compared with a range of existing IE models. Specifically, on this nursing handover data set, the macro-average F1 score from our model was 0.438, whilst the previous best deep learning models had 0.416. In conclusion, we showed that BERT based pre-training models can be transferred to health-related documents under mild conditions and with a proper fine-tuning process.
Comments: 6 pages, 4 figures
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2109.07243 [cs.CL]
  (or arXiv:2109.07243v2 [cs.CL] for this version)

Submission history

From: Chenchen Xu [view email]
[v1] Wed, 15 Sep 2021 12:22:57 GMT (4350kb,D)
[v2] Wed, 22 Sep 2021 12:00:19 GMT (4356kb,D)

Link back to: arXiv, form interface, contact.