References & Citations
Statistics > Machine Learning
Title: Truncated Variational Sampling for "Black Box" Optimization of Generative Models
(Submitted on 21 Dec 2017 (this version), latest version 22 Feb 2018 (v2))
Abstract: We investigate the optimization of two generative models with binary hidden variables using a novel variational EM approach. The novel approach distinguishes itself from previous variational approaches by using hidden states as variational parameters. Here we use efficient and general purpose sampling procedures to vary the hidden states, and investigate the "black box" applicability of the resulting optimization procedure. For general purpose applicability, samples are drawn from approximate marginal distributions of the considered generative model and from the prior distribution of a given generative model. As such, sampling is defined in a generic form with no additional derivations required. As a proof of concept, we then apply the novel procedure (A) to Binary Sparse Coding (a model with continuous observables), and (B) to basic Sigmoid Belief Networks (which are models with binary observables). The approach is applicable without any further analytical steps and efficiently as well as effectively increases the variational free-energy objective.
Submission history
From: Jörg Lücke [view email][v1] Thu, 21 Dec 2017 17:38:34 GMT (1446kb,D)
[v2] Thu, 22 Feb 2018 20:38:07 GMT (1457kb,D)
Link back to: arXiv, form interface, contact.