We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Information Retrieval

Title: Improving reference mining in patents with BERT

Abstract: In this paper we address the challenge of extracting scientific references from patents. We approach the problem as a sequence labelling task and investigate the merits of BERT models to the extraction of these long sequences. References in patents to scientific literature are relevant to study the connection between science and industry. Most prior work only uses the front-page citations for this analysis, which are provided in the metadata of patent archives. In this paper we build on prior work using Conditional Random Fields (CRF) and Flair for reference extraction. We improve the quality of the training data and train three BERT-based models on the labelled data (BERT, bioBERT, sciBERT). We find that the improved training data leads to a large improvement in the quality of the trained models. In addition, the BERT models beat CRF and Flair, with recall scores around 97% obtained with cross validation. With the best model we label a large collection of 33 thousand patents, extract the citations, and match them to publications in the Web of Science database. We extract 50% more references than with the old training data and methods: 735 thousand references in total. With these patent-publication links, follow-up research will further analyze which types of scientific work lead to inventions.
Comments: 10 pages, 3 figures
Subjects: Information Retrieval (cs.IR); Computation and Language (cs.CL)
ACM classes: H.3.1; I.2.7
Journal reference: Published in the 11th International Workshop on Bibliometric-enhanced Information Retrieval (BIR 2021)
Cite as: arXiv:2101.01039 [cs.IR]
  (or arXiv:2101.01039v3 [cs.IR] for this version)

Submission history

From: Suzan Verberne [view email]
[v1] Mon, 4 Jan 2021 15:56:21 GMT (384kb,D)
[v2] Fri, 15 Jan 2021 10:03:15 GMT (381kb,D)
[v3] Wed, 10 Mar 2021 11:26:01 GMT (192kb,D)

Link back to: arXiv, form interface, contact.