We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Tighter expected generalization error bounds via Wasserstein distance

Abstract: This work presents several expected generalization error bounds based on the Wasserstein distance. More specifically, it introduces full-dataset, single-letter, and random-subset bounds, and their analogues in the randomized subsample setting from Steinke and Zakynthinou [1]. Moreover, when the loss function is bounded and the geometry of the space is ignored by the choice of the metric in the Wasserstein distance, these bounds recover from below (and thus, are tighter than) current bounds based on the relative entropy. In particular, they generate new, non-vacuous bounds based on the relative entropy. Therefore, these results can be seen as a bridge between works that account for the geometry of the hypothesis space and those based on the relative entropy, which is agnostic to such geometry. Furthermore, it is shown how to produce various new bounds based on different information measures (e.g., the lautum information or several $f$-divergences) based on these bounds and how to derive similar bounds with respect to the backward channel using the presented proof techniques.
Comments: 29 pages: 9 of the main text, 3 of references, and 17 of appendices. Presented at ITR3 at ICML 2021. Accepted at NeurIPS 2021
Subjects: Machine Learning (stat.ML); Information Theory (cs.IT); Machine Learning (cs.LG)
Cite as: arXiv:2101.09315 [stat.ML]
  (or arXiv:2101.09315v2 [stat.ML] for this version)

Submission history

From: Borja Rodríguez Gálvez [view email]
[v1] Fri, 22 Jan 2021 20:13:59 GMT (311kb)
[v2] Fri, 25 Mar 2022 21:55:00 GMT (1382kb,D)

Link back to: arXiv, form interface, contact.