We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.CO

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Computation

Title: Monotonic Alpha-divergence Minimisation for Variational Inference

Abstract: In this paper, we introduce a novel family of iterative algorithms which carry out $\alpha$-divergence minimisation in a Variational Inference context. They do so by ensuring a systematic decrease at each step in the $\alpha$-divergence between the variational and the posterior distributions. In its most general form, the variational distribution is a mixture model and our framework allows us to simultaneously optimise the weights and components parameters of this mixture model. Notably, our approach permits to build on various methods previously proposed for $\alpha$-divergence minimisation such as Gradient or Power Descent schemes and we also shed a new light on an integrated Expectation Maximization algorithm. Lastly, we provide empirical evidence that our methodology yields improved results on several multimodal target distributions.
Subjects: Computation (stat.CO); Machine Learning (cs.LG); Statistics Theory (math.ST)
Cite as: arXiv:2103.05684 [stat.CO]
  (or arXiv:2103.05684v3 [stat.CO] for this version)

Submission history

From: Kamélia Daudel [view email]
[v1] Tue, 9 Mar 2021 19:41:03 GMT (1826kb,D)
[v2] Thu, 27 Jan 2022 18:12:11 GMT (14303kb,D)
[v3] Mon, 6 Jun 2022 14:45:29 GMT (19185kb,D)

Link back to: arXiv, form interface, contact.