We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Improving predictions of Bayesian neural networks via local linearization

Abstract: In this paper we argue that in Bayesian deep learning, the frequently utilized generalized Gauss-Newton (GGN) approximation should be understood as a modification of the underlying probabilistic model and should be considered separately from further approximate inference techniques. Applying the GGN approximation turns a BNN into a locally linearized generalized linear model or, equivalently, a Gaussian process. Because we then use this linearized model for inference, we should also predict using this modified likelihood rather than the original BNN likelihood. This formulation extends previous results to general likelihoods and alleviates underfitting behaviour observed e.g. by Ritter et al. (2018). We demonstrate our approach on several UCI classification datasets as well as CIFAR10.
Comments: ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:2008.08400 [stat.ML]
  (or arXiv:2008.08400v1 [stat.ML] for this version)

Submission history

From: Alexander Immer [view email]
[v1] Wed, 19 Aug 2020 12:35:55 GMT (4828kb,D)
[v2] Thu, 22 Oct 2020 14:45:01 GMT (4629kb,D)
[v3] Thu, 25 Feb 2021 17:59:47 GMT (5493kb,D)

Link back to: arXiv, form interface, contact.