Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Iterated Vector Fields and Conservatism, with Applications to Federated Learning
(Submitted on 8 Sep 2021 (v1), last revised 13 Nov 2021 (this version, v2))
Abstract: We study whether iterated vector fields (vector fields composed with themselves) are conservative. We give explicit examples of vector fields for which this self-composition preserves conservatism. Notably, this includes gradient vector fields of loss functions associated with some generalized linear models. As we show, characterizing the set of vector fields satisfying this condition leads to non-trivial geometric questions. In the context of federated learning, we show that when clients have loss functions whose gradients satisfy this condition, federated averaging is equivalent to gradient descent on a surrogate loss function. We leverage this to derive novel convergence results for federated learning. By contrast, we demonstrate that when the client losses violate this property, federated averaging can yield behavior which is fundamentally distinct from centralized optimization. Finally, we discuss theoretical and practical questions our analytical framework raises for federated learning.
Submission history
From: Zachary Charles [view email][v1] Wed, 8 Sep 2021 23:31:54 GMT (20kb)
[v2] Sat, 13 Nov 2021 00:22:24 GMT (42kb,D)
Link back to: arXiv, form interface, contact.