We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Machine Learning

Title: Variational Representations of Annealing Paths: Bregman Information under Monotonic Embedding

Abstract: Markov Chain Monte Carlo methods for sampling from complex distributions and estimating normalization constants often simulate samples from a sequence of intermediate distributions along an annealing path, which bridges between a tractable initial distribution and a target density of interest. Prior works have constructed annealing paths using quasi-arithmetic means, and interpreted the resulting intermediate densities as minimizing an expected divergence to the endpoints. To analyze these variational representations of annealing paths, we extend known results showing that the arithmetic mean over arguments minimizes the expected Bregman divergence to a single representative point. In particular, we obtain an analogous result for quasi-arithmetic means, when the inputs to the Bregman divergence are transformed under a monotonic embedding function. Our analysis highlights the interplay between quasi-arithmetic means, parametric families, and divergence functionals using the rho-tau representational Bregman divergence framework, and associates common divergence functionals with intermediate densities along an annealing path.
Comments: Published in Information Geometry (Info. Geo. 2024)
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT); Statistics Theory (math.ST); Machine Learning (stat.ML)
DOI: 10.1007/s41884-023-00129-6
Cite as: arXiv:2209.07481 [cs.LG]
  (or arXiv:2209.07481v3 [cs.LG] for this version)

Submission history

From: Rob Brekelmans [view email]
[v1] Thu, 15 Sep 2022 17:22:04 GMT (1579kb)
[v2] Thu, 12 Oct 2023 17:59:22 GMT (2099kb,D)
[v3] Tue, 6 Feb 2024 13:35:14 GMT (643kb,D)

Link back to: arXiv, form interface, contact.