Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Faithful Model Inversion Substantially Improves Auto-encoding Variational Inference
(Submitted on 1 Dec 2017 (this version), latest version 29 Nov 2018 (v5))
Abstract: In learning deep generative models, the encoder for variational inference is typically formed in an ad hoc manner with a structure and parametrization analogous to the forward model. Our chief insight is that this results in coarse approximations to the posterior, and that the d-separation properties of the BN structure of the forward model should be used, in a principled way, to produce ones that are faithful to the posterior, for which we introduce the novel Compact Minimal I-map (CoMI) algorithm. Applying our method to common models reveals that standard encoder design choices lack many important edges, and through experiments we demonstrate that modelling these edges is important for optimal learning. We show how using a faithful encoder is crucial when modelling with continuous relaxations of categorical distributions.
Submission history
From: Stefan Webb [view email][v1] Fri, 1 Dec 2017 12:08:03 GMT (5554kb,D)
[v2] Sat, 10 Feb 2018 18:16:17 GMT (8849kb,D)
[v3] Tue, 29 May 2018 15:47:44 GMT (8244kb,D)
[v4] Wed, 24 Oct 2018 15:58:18 GMT (9104kb,D)
[v5] Thu, 29 Nov 2018 14:27:47 GMT (9087kb,D)
Link back to: arXiv, form interface, contact.