We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: FERMI: Fair Empirical Risk Minimization via Exponential Rényi Mutual Information

Abstract: Despite the success of large-scale empirical risk minimization (ERM) at achieving high accuracy across a variety of machine learning tasks, fair ERM is hindered by the incompatibility of fairness constraints with stochastic optimization. In this paper, we propose the fair empirical risk minimization via exponential R\'enyi mutual information (FERMI) framework. FERMI is built on a stochastic estimator for exponential R\'enyi mutual information (ERMI), an information divergence measuring the degree of the dependence of predictions on sensitive attributes. Theoretically, we show that ERMI upper bounds existing popular fairness violation metrics, thus controlling ERMI provides guarantees on other commonly used violations, such as $L_\infty$. We derive an unbiased estimator for ERMI, which we use to derive the FERMI algorithm. We prove that FERMI converges for demographic parity, equalized odds, and equal opportunity notions of fairness in stochastic optimization. Empirically, we show that FERMI is amenable to large-scale problems with multiple (non-binary) sensitive attributes and non-binary targets. Extensive experiments show that FERMI achieves the most favorable tradeoffs between fairness violation and test accuracy across all tested setups compared with state-of-the-art baselines for demographic parity, equalized odds, equal opportunity. These benefits are especially significant for non-binary classification with large sensitive sets and small batch sizes, showcasing the effectiveness of the FERMI objective and the developed stochastic algorithm for solving it.
Comments: 29 pages
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT)
Cite as: arXiv:2102.12586 [cs.LG]
  (or arXiv:2102.12586v2 [cs.LG] for this version)

Submission history

From: Ahmad Beirami [view email]
[v1] Wed, 24 Feb 2021 22:15:44 GMT (2781kb,D)
[v2] Sun, 25 Jul 2021 22:22:51 GMT (7130kb,D)

Link back to: arXiv, form interface, contact.