Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Diagnostic Uncertainty Calibration: Towards Reliable Machine Predictions in Medical Domain
(Submitted on 3 Jul 2020 (v1), last revised 22 Mar 2021 (this version, v4))
Abstract: We propose an evaluation framework for class probability estimates (CPEs) in the presence of label uncertainty, which is commonly observed as diagnosis disagreement between experts in the medical domain. We also formalize evaluation metrics for higher-order statistics, including inter-rater disagreement, to assess predictions on label uncertainty. Moreover, we propose a novel post-hoc method called $alpha$-calibration, that equips neural network classifiers with calibrated distributions over CPEs. Using synthetic experiments and a large-scale medical imaging application, we show that our approach significantly enhances the reliability of uncertainty estimates: disagreement probabilities and posterior CPEs.
Submission history
From: Takahiro Mimori [view email][v1] Fri, 3 Jul 2020 12:54:08 GMT (372kb,D)
[v2] Tue, 7 Jul 2020 02:07:20 GMT (371kb,D)
[v3] Mon, 25 Jan 2021 04:40:25 GMT (554kb,D)
[v4] Mon, 22 Mar 2021 06:23:53 GMT (832kb,D)
Link back to: arXiv, form interface, contact.