Current browse context:
stat.CO
Change to browse by:
References & Citations
Statistics > Computation
Title: Monotonic Alpha-divergence Minimisation for Variational Inference
(Submitted on 9 Mar 2021 (v1), last revised 10 Apr 2023 (this version, v4))
Abstract: In this paper, we introduce a novel family of iterative algorithms which carry out $\alpha$-divergence minimisation in a Variational Inference context. They do so by ensuring a systematic decrease at each step in the $\alpha$-divergence between the variational and the posterior distributions. In its most general form, the variational distribution is a mixture model and our framework allows us to simultaneously optimise the weights and components parameters of this mixture model. Our approach permits us to build on various methods previously proposed for $\alpha$-divergence minimisation such as Gradient or Power Descent schemes and we also shed a new light on an integrated Expectation Maximization algorithm. Lastly, we provide empirical evidence that our methodology yields improved results on several multimodal target distributions and on a real data example.
Submission history
From: Kamélia Daudel [view email][v1] Tue, 9 Mar 2021 19:41:03 GMT (1826kb,D)
[v2] Thu, 27 Jan 2022 18:12:11 GMT (14303kb,D)
[v3] Mon, 6 Jun 2022 14:45:29 GMT (19185kb,D)
[v4] Mon, 10 Apr 2023 21:12:21 GMT (15915kb,D)
Link back to: arXiv, form interface, contact.