Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Fast Online Value-Maximizing Prediction Sets with Conformal Cost Control
(Submitted on 2 Feb 2023 (v1), last revised 25 Apr 2023 (this version, v3))
Abstract: Many real-world multi-label prediction problems involve set-valued predictions that must satisfy specific requirements dictated by downstream usage. We focus on a typical scenario where such requirements, separately encoding $\textit{value}$ and $\textit{cost}$, compete with each other. For instance, a hospital might expect a smart diagnosis system to capture as many severe, often co-morbid, diseases as possible (the value), while maintaining strict control over incorrect predictions (the cost). We present a general pipeline, dubbed as FavMac, to maximize the value while controlling the cost in such scenarios. FavMac can be combined with almost any multi-label classifier, affording distribution-free theoretical guarantees on cost control. Moreover, unlike prior works, it can handle real-world large-scale applications via a carefully designed online update mechanism, which is of independent interest. Our methodological and theoretical contributions are supported by experiments on several healthcare tasks and synthetic datasets - FavMac furnishes higher value compared with several variants and baselines while maintaining strict cost control. Our code is available at this https URL
Submission history
From: Shubhendu Trivedi [view email][v1] Thu, 2 Feb 2023 02:53:12 GMT (88kb,D)
[v2] Sat, 18 Feb 2023 05:53:51 GMT (88kb,D)
[v3] Tue, 25 Apr 2023 17:48:32 GMT (96kb,D)
Link back to: arXiv, form interface, contact.