References & Citations
Computer Science > Machine Learning
Title: Personalized Federated Learning with Contextualized Generalization
(Submitted on 24 Jun 2021 (v1), last revised 1 May 2022 (this version, v2))
Abstract: The prevalent personalized federated learning (PFL) usually pursues a trade-off between personalization and generalization by maintaining a shared global model to guide the training process of local models. However, the sole global model may easily transfer deviated context knowledge to some local models when multiple latent contexts exist across the local datasets. In this paper, we propose a novel concept called contextualized generalization (CG) to provide each client with fine-grained context knowledge that can better fit the local data distributions and facilitate faster model convergence, based on which we properly design a framework of PFL, dubbed CGPFL. We conduct detailed theoretical analysis, in which the convergence guarantee is presented and $\mathcal{O}(\sqrt{K})$ speedup over most existing methods is granted. To quantitatively study the generalization-personalization trade-off, we introduce the 'generalization error' measure and prove that the proposed CGPFL can achieve a better trade-off than existing solutions. Moreover, our theoretical analysis further inspires a heuristic algorithm to find a near-optimal trade-off in CGPFL. Experimental results on multiple real-world datasets show that our approach surpasses the state-of-the-art methods on test accuracy by a significant margin.
Submission history
From: Xueyang Tang [view email][v1] Thu, 24 Jun 2021 14:17:00 GMT (1925kb,D)
[v2] Sun, 1 May 2022 08:19:28 GMT (4242kb,D)
Link back to: arXiv, form interface, contact.