Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Training Uncertainty-Aware Classifiers with Conformalized Deep Learning
(Submitted on 12 May 2022 (v1), last revised 9 Nov 2022 (this version, v2))
Abstract: Deep neural networks are powerful tools to detect hidden patterns in data and leverage them to make predictions, but they are not designed to understand uncertainty and estimate reliable probabilities. In particular, they tend to be overconfident. We begin to address this problem in the context of multi-class classification by developing a novel training algorithm producing models with more dependable uncertainty estimates, without sacrificing predictive power. The idea is to mitigate overconfidence by minimizing a loss function, inspired by advances in conformal inference, that quantifies model uncertainty by carefully leveraging hold-out data. Experiments with synthetic and real data demonstrate this method can lead to smaller conformal prediction sets with higher conditional coverage, after exact calibration with hold-out data, compared to state-of-the-art alternatives.
Submission history
From: Matteo Sesia [view email][v1] Thu, 12 May 2022 05:08:10 GMT (10032kb,D)
[v2] Wed, 9 Nov 2022 03:16:20 GMT (11929kb,D)
Link back to: arXiv, form interface, contact.