Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: $(f,Γ)$-Divergences: Interpolating between $f$-Divergences and Integral Probability Metrics
(Submitted on 11 Nov 2020 (v1), last revised 15 Sep 2021 (this version, v3))
Abstract: We develop a rigorous and general framework for constructing information-theoretic divergences that subsume both $f$-divergences and integral probability metrics (IPMs), such as the $1$-Wasserstein distance. We prove under which assumptions these divergences, hereafter referred to as $(f,\Gamma)$-divergences, provide a notion of `distance' between probability measures and show that they can be expressed as a two-stage mass-redistribution/mass-transport process. The $(f,\Gamma)$-divergences inherit features from IPMs, such as the ability to compare distributions which are not absolutely continuous, as well as from $f$-divergences, namely the strict concavity of their variational representations and the ability to control heavy-tailed distributions for particular choices of $f$. When combined, these features establish a divergence with improved properties for estimation, statistical learning, and uncertainty quantification applications. Using statistical learning as an example, we demonstrate their advantage in training generative adversarial networks (GANs) for heavy-tailed, not-absolutely continuous sample distributions. We also show improved performance and stability over gradient-penalized Wasserstein GAN in image generation.
Submission history
From: Jeremiah Birrell [view email][v1] Wed, 11 Nov 2020 18:17:09 GMT (956kb)
[v2] Wed, 27 Jan 2021 19:21:56 GMT (1932kb)
[v3] Wed, 15 Sep 2021 14:25:24 GMT (4185kb)
Link back to: arXiv, form interface, contact.