Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Joint Stochastic Approximation learning of Helmholtz Machines
(Submitted on 20 Mar 2016 (v1), last revised 28 Sep 2018 (this version, v2))
Abstract: Though with progress, model learning and performing posterior inference still remains a common challenge for using deep generative models, especially for handling discrete hidden variables. This paper is mainly concerned with algorithms for learning Helmholz machines, which is characterized by pairing the generative model with an auxiliary inference model. A common drawback of previous learning algorithms is that they indirectly optimize some bounds of the targeted marginal log-likelihood. In contrast, we successfully develop a new class of algorithms, based on stochastic approximation (SA) theory of the Robbins-Monro type, to directly optimize the marginal log-likelihood and simultaneously minimize the inclusive KL-divergence. The resulting learning algorithm is thus called joint SA (JSA). Moreover, we construct an effective MCMC operator for JSA. Our results on the MNIST datasets demonstrate that the JSA's performance is consistently superior to that of competing algorithms like RWS, for learning a range of difficult models.
Submission history
From: Zhijian Ou [view email][v1] Sun, 20 Mar 2016 00:55:06 GMT (693kb)
[v2] Fri, 28 Sep 2018 13:16:25 GMT (767kb)
Link back to: arXiv, form interface, contact.