Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: MURANA: A Generic Framework for Stochastic Variance-Reduced Optimization
(Submitted on 6 Jun 2021 (v1), revised 7 Mar 2022 (this version, v2), latest version 6 Mar 2023 (v3))
Abstract: We propose a generic variance-reduced algorithm, which we call MUltiple RANdomized Algorithm (MURANA), for minimizing a sum of several smooth functions plus a regularizer, in a sequential or distributed manner. Our method is formulated with general stochastic operators, which allow us to model various strategies for reducing the computational complexity. For example, MURANA supports sparse activation of the gradients, and also reduction of the communication load via compression of the update vectors. This versatility allows MURANA to cover many existing randomization mechanisms within a unified framework. However, MURANA also encodes new methods as special cases. We highlight one of them, which we call ELVIRA, and show that it improves upon Loopless SVRG.
Submission history
From: Laurent Condat [view email][v1] Sun, 6 Jun 2021 07:39:01 GMT (18kb)
[v2] Mon, 7 Mar 2022 09:54:40 GMT (20kb)
[v3] Mon, 6 Mar 2023 13:39:15 GMT (1114kb)
Link back to: arXiv, form interface, contact.