We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Nonparametric Hamiltonian Monte Carlo

Abstract: Probabilistic programming uses programs to express generative models whose posterior probability is then computed by built-in inference engines. A challenging goal is to develop general purpose inference algorithms that work out-of-the-box for arbitrary programs in a universal probabilistic programming language (PPL). The densities defined by such programs, which may use stochastic branching and recursion, are (in general) nonparametric, in the sense that they correspond to models on an infinite-dimensional parameter space. However standard inference algorithms, such as the Hamiltonian Monte Carlo (HMC) algorithm, target distributions with a fixed number of parameters. This paper introduces the Nonparametric Hamiltonian Monte Carlo (NP-HMC) algorithm which generalises HMC to nonparametric models. Inputs to NP-HMC are a new class of measurable functions called "tree representable", which serve as a language-independent representation of the density functions of probabilistic programs in a universal PPL. We provide a correctness proof of NP-HMC, and empirically demonstrate significant performance improvements over existing approaches on several nonparametric examples.
Comments: Updated plots (after fixing minor bugs in the implementation) compared to the published version in Proceedings of the 38th International Conference on Machine Learning, PMLR 139, 2021. The conclusions of the version published at ICML 2021 are not affected
Subjects: Machine Learning (cs.LG); Programming Languages (cs.PL); Computation (stat.CO); Machine Learning (stat.ML)
Cite as: arXiv:2106.10238 [cs.LG]
  (or arXiv:2106.10238v2 [cs.LG] for this version)

Submission history

From: Fabian Zaiser [view email]
[v1] Fri, 18 Jun 2021 17:03:05 GMT (3840kb,D)
[v2] Tue, 1 Nov 2022 18:50:54 GMT (3822kb,D)

Link back to: arXiv, form interface, contact.