Current browse context:
stat.ML
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title: Human-centric Metric for Accelerating Pathology Reports Annotation
(Submitted on 31 Oct 2019 (v1), last revised 12 Nov 2019 (this version, v2))
Abstract: Pathology reports contain useful information such as the main involved organ, diagnosis, etc. These information can be identified from the free text reports and used for large-scale statistical analysis or serve as annotation for other modalities such as pathology slides images. However, manual classification for a huge number of reports on multiple tasks is labor-intensive. In this paper, we have developed an automatic text classifier based on BERT and we propose a human-centric metric to evaluate the model. According to the model confidence, we identify low-confidence cases that require further expert annotation and high-confidence cases that are automatically classified. We report the percentage of low-confidence cases and the performance of automatically classified cases. On the high-confidence cases, the model achieves classification accuracy comparable to pathologists. This leads a potential of reducing 80% to 98% of the manual annotation workload.
Submission history
From: Ruibin Ma [view email][v1] Thu, 31 Oct 2019 22:09:19 GMT (188kb,D)
[v2] Tue, 12 Nov 2019 15:12:45 GMT (191kb,D)
Link back to: arXiv, form interface, contact.