Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: A Decentralized Approach to Bayesian Learning
(Submitted on 14 Jul 2020 (v1), last revised 9 Jan 2021 (this version, v4))
Abstract: Motivated by decentralized approaches to machine learning, we propose a collaborative Bayesian learning algorithm taking the form of decentralized Langevin dynamics in a non-convex setting. Our analysis show that the initial KL-divergence between the Markov Chain and the target posterior distribution is exponentially decreasing while the error contributions to the overall KL-divergence from the additive noise is decreasing in polynomial time. We further show that the polynomial-term experiences speed-up with number of agents and provide sufficient conditions on the time-varying step-sizes to guarantee convergence to the desired distribution. The performance of the proposed algorithm is evaluated on a wide variety of machine learning tasks. The empirical results show that the performance of individual agents with locally available data is on par with the centralized setting with considerable improvement in the convergence rate.
Submission history
From: Anjaly Parayil [view email][v1] Tue, 14 Jul 2020 03:59:17 GMT (1905kb)
[v2] Fri, 23 Oct 2020 16:33:20 GMT (2298kb,D)
[v3] Mon, 2 Nov 2020 18:08:40 GMT (2298kb,D)
[v4] Sat, 9 Jan 2021 17:01:44 GMT (4624kb,D)
Link back to: arXiv, form interface, contact.