Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Faster Convergence of Stochastic Gradient Langevin Dynamics for Non-Log-Concave Sampling
(Submitted on 19 Oct 2020 (v1), last revised 23 Feb 2021 (this version, v2))
Abstract: We provide a new convergence analysis of stochastic gradient Langevin dynamics (SGLD) for sampling from a class of distributions that can be non-log-concave. At the core of our approach is a novel conductance analysis of SGLD using an auxiliary time-reversible Markov Chain. Under certain conditions on the target distribution, we prove that $\tilde O(d^4\epsilon^{-2})$ stochastic gradient evaluations suffice to guarantee $\epsilon$-sampling error in terms of the total variation distance, where $d$ is the problem dimension. This improves existing results on the convergence rate of SGLD (Raginsky et al., 2017; Xu et al., 2018). We further show that provided an additional Hessian Lipschitz condition on the log-density function, SGLD is guaranteed to achieve $\epsilon$-sampling error within $\tilde O(d^{15/4}\epsilon^{-3/2})$ stochastic gradient evaluations. Our proof technique provides a new way to study the convergence of Langevin-based algorithms and sheds some light on the design of fast stochastic gradient-based sampling algorithms.
Submission history
From: Quanquan Gu [view email][v1] Mon, 19 Oct 2020 15:23:18 GMT (112kb,D)
[v2] Tue, 23 Feb 2021 07:15:34 GMT (171kb,D)
Link back to: arXiv, form interface, contact.