We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: PAC-Bayesian Theory Meets Bayesian Inference

Authors: Pascal Germain (INRIA Paris), Francis Bach (INRIA Paris), Alexandre Lacoste (Google), Simon Lacoste-Julien (INRIA Paris)
Abstract: We exhibit a strong link between frequentist PAC-Bayesian bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam's razor criteria, under the assumption that the data is generated by a i.i.d. distribution. Moreover, as the negative log-likelihood is an unbounded loss function, we motivate and propose a PAC-Bayesian theorem tailored for the sub-Gamma loss family, and we show that our approach is sound on classical Bayesian linear regression tasks.
Comments: under review
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:1605.08636 [stat.ML]
  (or arXiv:1605.08636v1 [stat.ML] for this version)

Submission history

From: Pascal Germain [view email]
[v1] Fri, 27 May 2016 13:41:33 GMT (525kb,D)
[v2] Tue, 1 Nov 2016 14:49:05 GMT (530kb,D)
[v3] Sat, 3 Dec 2016 22:48:15 GMT (529kb,D)
[v4] Mon, 13 Feb 2017 17:14:52 GMT (530kb,D)

Link back to: arXiv, form interface, contact.