Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: The $χ$-Divergence for Approximate Inference
(Submitted on 1 Nov 2016 (v1), revised 27 Feb 2017 (this version, v2), latest version 12 Nov 2017 (v4))
Abstract: Variational inference enables Bayesian analysis for complex probabilistic models with massive data sets. It posits a family of approximating distributions and finds the member closest to the posterior. While successful, variational inference methods can run into pathologies; for example, they typically underestimate posterior uncertainty. In this paper we propose CHIVI, a complementary algorithm to traditional variational inference. CHIVI is a black box algorithm that minimizes the $\chi$-divergence from the posterior to the family of approximating distributions and provides an upper bound of the model evidence. We studied CHIVI in several scenarios. On Bayesian probit regression and Gaussian process classification it yielded better classification error rates than expectation propagation (EP) and classical variational inference (VI). When modeling basketball data with a Cox process, it gave better estimates of posterior uncertainty. Finally, we show how to use the CHIVI upper bound and classical VI lower bound to sandwich estimate the model evidence.
Submission history
From: Adji Bousso Dieng [view email][v1] Tue, 1 Nov 2016 18:40:23 GMT (6045kb,D)
[v2] Mon, 27 Feb 2017 03:00:03 GMT (5663kb,D)
[v3] Mon, 6 Nov 2017 00:29:21 GMT (3841kb,D)
[v4] Sun, 12 Nov 2017 19:00:57 GMT (7729kb,D)
Link back to: arXiv, form interface, contact.