We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Truncated Variational Sampling for "Black Box" Optimization of Generative Models

Abstract: We investigate the optimization of two probabilistic generative models with binary latent variables using a novel variational EM approach. The approach distinguishes itself from previous variational approaches by using latent states as variational parameters. Here we use efficient and general purpose sampling procedures to vary the latent states, and investigate the "black box" applicability of the resulting optimization procedure. For general purpose applicability, samples are drawn from approximate marginal distributions of the considered generative model as well as from the model's prior distribution. As such, variational sampling is defined in a generic form, and is directly executable for a given model. As a proof of concept, we then apply the novel procedure (A) to Binary Sparse Coding (a model with continuous observables), and (B) to basic Sigmoid Belief Networks (which are models with binary observables). Numerical experiments verify that the investigated approach efficiently as well as effectively increases a variational free energy objective without requiring any additional analytical steps.
Subjects: Machine Learning (stat.ML)
Cite as: arXiv:1712.08104 [stat.ML]
  (or arXiv:1712.08104v2 [stat.ML] for this version)

Submission history

From: Jörg Lücke [view email]
[v1] Thu, 21 Dec 2017 17:38:34 GMT (1446kb,D)
[v2] Thu, 22 Feb 2018 20:38:07 GMT (1457kb,D)

Link back to: arXiv, form interface, contact.