References & Citations
Computer Science > Machine Learning
Title: Proportional Multicalibration
(Submitted on 29 Sep 2022 (this version), latest version 31 Aug 2023 (v3))
Abstract: Multicalibration is a desirable fairness criteria that constrains calibration error among flexibly-defined groups in the data while maintaining overall calibration. However, when outcome probabilities are correlated with group membership, multicalibrated models can exhibit a higher percent calibration error among groups with lower base rates than groups with higher base rates. As a result, it remains possible for a decision-maker to learn to trust or distrust model predictions for specific groups. To alleviate this, we propose proportional multicalibration, a criteria that constrains the percent calibration error among groups and within prediction bins. We prove that satisfying proportional multicalibration bounds a model's multicalibration as well its differential calibration, a stronger fairness criteria inspired by the fairness notion of sufficiency. We provide an efficient algorithm for post-processing risk prediction models for proportional multicalibration and evaluate it empirically. We conduct simulation studies and investigate a real-world application of PMC-postprocessing to prediction of emergency department patient admissions. We observe that proportional multicalibration is a promising criteria for controlling simultenous measures of calibration fairness of a model over intersectional groups with virtually no cost in terms of classification performance.
Submission history
From: William La Cava [view email][v1] Thu, 29 Sep 2022 08:15:29 GMT (5656kb,D)
[v2] Thu, 5 Jan 2023 21:11:15 GMT (5687kb,D)
[v3] Thu, 31 Aug 2023 19:57:50 GMT (8211kb,D)
Link back to: arXiv, form interface, contact.