Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Approximation Based Variance Reduction for Reparameterization Gradients
(Submitted on 29 Jul 2020 (v1), last revised 23 Oct 2020 (this version, v2))
Abstract: Flexible variational distributions improve variational inference but are harder to optimize. In this work we present a control variate that is applicable for any reparameterizable distribution with known mean and covariance matrix, e.g. Gaussians with any covariance structure. The control variate is based on a quadratic approximation of the model, and its parameters are set using a double-descent scheme by minimizing the gradient estimator's variance. We empirically show that this control variate leads to large improvements in gradient variance and optimization convergence for inference with non-factorized variational distributions.
Submission history
From: Tomas Geffner [view email][v1] Wed, 29 Jul 2020 06:55:11 GMT (1008kb,D)
[v2] Fri, 23 Oct 2020 08:36:43 GMT (2167kb,D)
Link back to: arXiv, form interface, contact.