Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Predictive Multiplicity in Classification
(Submitted on 14 Sep 2019 (v1), last revised 16 Sep 2020 (this version, v4))
Abstract: Prediction problems often admit competing models that perform almost equally well. This effect challenges key assumptions in machine learning when competing models assign conflicting predictions. In this paper, we define predictive multiplicity as the ability of a prediction problem to admit competing models with conflicting predictions. We introduce formal measures to evaluate the severity of predictive multiplicity and develop integer programming tools to compute them exactly for linear classification problems. We apply our tools to measure predictive multiplicity in recidivism prediction problems. Our results show that real-world datasets may admit competing models that assign wildly conflicting predictions, and motivate the need to measure and report predictive multiplicity in model development.
Submission history
From: Charles Marx [view email][v1] Sat, 14 Sep 2019 21:12:04 GMT (4073kb,D)
[v2] Thu, 12 Dec 2019 08:39:47 GMT (5373kb,D)
[v3] Mon, 13 Jul 2020 05:10:08 GMT (3989kb,D)
[v4] Wed, 16 Sep 2020 04:58:29 GMT (3765kb,D)
Link back to: arXiv, form interface, contact.