References & Citations
Statistics > Machine Learning
Title: Extreme Stochastic Variational Inference: Distributed and Asynchronous
(Submitted on 31 May 2016 (v1), revised 22 Jun 2016 (this version, v2), latest version 3 Aug 2018 (v9))
Abstract: We propose extreme stochastic variational inference (ESVI), which allows multiple processors to simultaneously and asynchronously perform variational inference updates. Moreover, by using a classic owner computes paradigm, our algorithm can be made lock-free. ESVI exhibits data and model parallelism, that is, each processor only needs access to a subset of the data and a subset of the parameters. In our experiments we show that our new algorithm outperforms a straightforward strategy for parallelizing variational inference, which requires bulk synchronization after every iteration.
Submission history
From: Parameswaran Raman [view email][v1] Tue, 31 May 2016 05:10:51 GMT (2298kb,D)
[v2] Wed, 22 Jun 2016 05:55:09 GMT (2298kb,D)
[v3] Wed, 15 Mar 2017 20:50:37 GMT (2581kb,D)
[v4] Fri, 21 Apr 2017 17:37:39 GMT (2692kb,D)
[v5] Tue, 23 May 2017 08:14:27 GMT (1571kb,D)
[v6] Thu, 15 Feb 2018 01:02:49 GMT (904kb,D)
[v7] Wed, 18 Apr 2018 01:10:33 GMT (907kb,D)
[v8] Mon, 21 May 2018 23:39:58 GMT (907kb,D)
[v9] Fri, 3 Aug 2018 22:05:49 GMT (905kb,D)
Link back to: arXiv, form interface, contact.