We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: PAC-Bayes unleashed: generalisation bounds with unbounded losses

Abstract: We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems with a bounded loss function (typically assumed to take values in the interval [0;1]). In order to relax this assumption, we propose a new notion called HYPE (standing for \emph{HYPothesis-dependent rangE}), which effectively allows the range of the loss to depend on each predictor. Based on this new notion we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions.
Comments: 24 pages
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
Journal reference: Entropy 2021
DOI: 10.3390/e23101330
Cite as: arXiv:2006.07279 [stat.ML]
  (or arXiv:2006.07279v2 [stat.ML] for this version)

Submission history

From: Benjamin Guedj [view email]
[v1] Fri, 12 Jun 2020 15:55:46 GMT (61kb,D)
[v2] Wed, 30 Sep 2020 16:02:45 GMT (59kb,D)

Link back to: arXiv, form interface, contact.