We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Challenges in Bayesian inference via Markov chain Monte Carlo for neural networks

Abstract: Markov chain Monte Carlo (MCMC) methods and neural networks are instrumental in tackling inferential and prediction problems. However, Bayesian inference based on joint use of MCMC methods and of neural networks is limited. This paper reviews the main challenges posed by neural networks to MCMC developments, including lack of parameter identifiability due to weight symmetries, prior specification effects, and consequently high computational cost and convergence failure. Population and manifold MCMC algorithms are combined to demonstrate these challenges via multilayer perceptron (MLP) examples and to develop case studies for assessing the capacity of approximate inference methods to uncover the posterior covariance of neural network parameters. Some of these challenges, such as high computational cost arising from the application of neural networks to big data and parameter identifiability arising from weight symmetries, stimulate research towards more scalable approximate MCMC methods or towards MCMC methods in reduced parameter spaces.
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Computation (stat.CO); Methodology (stat.ME)
Cite as: arXiv:1910.06539 [stat.ML]
  (or arXiv:1910.06539v3 [stat.ML] for this version)

Submission history

From: Theodore Papamarkou [view email]
[v1] Tue, 15 Oct 2019 05:43:45 GMT (3253kb,D)
[v2] Tue, 22 Oct 2019 18:02:21 GMT (3267kb,D)
[v3] Thu, 14 Nov 2019 03:37:55 GMT (3267kb,D)

Link back to: arXiv, form interface, contact.