We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Quantifying Model Predictive Uncertainty with Perturbation Theory

Abstract: We propose a framework for predictive uncertainty quantification of a neural network that replaces the conventional Bayesian notion of weight probability density function (PDF) with a physics based potential field representation of the model weights in a Gaussian reproducing kernel Hilbert space (RKHS) embedding. This allows us to use perturbation theory from quantum physics to formulate a moment decomposition problem over the model weight-output relationship. The extracted moments reveal successive degrees of regularization of the weight PDF around the local neighborhood of the model output. Such localized moments determine with great sensitivity the local heterogeneity of the weight PDF around a model prediction thereby providing significantly greater accuracy of model predictive uncertainty than the central moments characterized by Bayesian and ensemble methods. We show that this consequently leads to a better ability to detect false model predictions of test data that has undergone a covariate shift away from the training PDF learned by the model. We evaluate our approach against baseline uncertainty quantification methods on several benchmark datasets that are corrupted using common distortion techniques. Our approach provides fast model predictive uncertainty estimates with much greater precision and calibration.
Comments: Section 3 added for notational clarity. Updated version of arXiv:2103.01374
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT)
Cite as: arXiv:2109.10888 [cs.LG]
  (or arXiv:2109.10888v2 [cs.LG] for this version)

Submission history

From: Rishabh Singh [view email]
[v1] Wed, 22 Sep 2021 17:55:09 GMT (16610kb,D)
[v2] Thu, 18 Nov 2021 18:57:41 GMT (16617kb,D)

Link back to: arXiv, form interface, contact.