Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC
(Submitted on 12 Jun 2020 (this version), latest version 16 Mar 2022 (v2))
Abstract: Learning energy-based model (EBM) requires MCMC sampling of the learned model as the inner loop of the learning algorithm. However, MCMC sampling of EBM in data space is generally not mixing, because the energy function, which is usually parametrized by deep network, is highly multi-modal in the data space. This is a serious handicap for both the theory and practice of EBM. In this paper, we propose to learn EBM with a flow-based model serving as a backbone, so that the EBM is a correction or an exponential tilting of the flow-based model. We show that the model has a particularly simple form in the space of the latent variables of the flow-based model, and MCMC sampling of the EBM in the latent space, which is a simple special case of neural transport MCMC, mixes well and traverses modes in the data space. This enables proper sampling and learning of EBM.
Submission history
From: Erik Nijkamp [view email][v1] Fri, 12 Jun 2020 01:25:51 GMT (9068kb,D)
[v2] Wed, 16 Mar 2022 07:53:07 GMT (9340kb,D)
Link back to: arXiv, form interface, contact.