We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Finetuning BERT on Partially Annotated NER Corpora

Abstract: Most Named Entity Recognition (NER) models operate under the assumption that training datasets are fully labelled. While it is valid for established datasets like CoNLL 2003 and OntoNotes, sometimes it is not feasible to obtain the complete dataset annotation. These situations may occur, for instance, after selective annotation of entities for cost reduction. This work presents an approach to finetuning BERT on such partially labelled datasets using self-supervision and label preprocessing. Our approach outperforms the previous LSTM-based label preprocessing baseline, significantly improving the performance on poorly labelled datasets. We demonstrate that following our approach while finetuning RoBERTa on CoNLL 2003 dataset with only 10% of total entities labelled is enough to reach the performance of the baseline trained on the same dataset with 50% of the entities labelled.
Comments: 6 pages, to be published in Proceedings of ISP RAS Open Conference 2022
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2211.14360 [cs.CL]
  (or arXiv:2211.14360v1 [cs.CL] for this version)

Submission history

From: Viktor Scherbakov [view email]
[v1] Fri, 25 Nov 2022 19:54:30 GMT (2434kb,D)

Link back to: arXiv, form interface, contact.