References & Citations
Computer Science > Information Retrieval
Title: Improving reference mining in patents with BERT
(Submitted on 4 Jan 2021 (v1), last revised 10 Mar 2021 (this version, v3))
Abstract: In this paper we address the challenge of extracting scientific references from patents. We approach the problem as a sequence labelling task and investigate the merits of BERT models to the extraction of these long sequences. References in patents to scientific literature are relevant to study the connection between science and industry. Most prior work only uses the front-page citations for this analysis, which are provided in the metadata of patent archives. In this paper we build on prior work using Conditional Random Fields (CRF) and Flair for reference extraction. We improve the quality of the training data and train three BERT-based models on the labelled data (BERT, bioBERT, sciBERT). We find that the improved training data leads to a large improvement in the quality of the trained models. In addition, the BERT models beat CRF and Flair, with recall scores around 97% obtained with cross validation. With the best model we label a large collection of 33 thousand patents, extract the citations, and match them to publications in the Web of Science database. We extract 50% more references than with the old training data and methods: 735 thousand references in total. With these patent-publication links, follow-up research will further analyze which types of scientific work lead to inventions.
Submission history
From: Suzan Verberne [view email][v1] Mon, 4 Jan 2021 15:56:21 GMT (384kb,D)
[v2] Fri, 15 Jan 2021 10:03:15 GMT (381kb,D)
[v3] Wed, 10 Mar 2021 11:26:01 GMT (192kb,D)
Link back to: arXiv, form interface, contact.