References & Citations
Computer Science > Machine Learning
Title: GPT-FL: Generative Pre-trained Model-Assisted Federated Learning
(Submitted on 3 Jun 2023 (v1), last revised 29 Sep 2023 (this version, v3))
Abstract: In this work, we propose GPT-FL, a generative pre-trained model-assisted federated learning (FL) framework. At its core, GPT-FL leverages generative pre-trained models to generate diversified synthetic data. These generated data are used to train a downstream model on the server, which is then fine-tuned with private client data under the standard FL framework. We show that GPT-FL consistently outperforms state-of-the-art FL methods in terms of model test accuracy, communication efficiency, and client sampling efficiency. Through comprehensive ablation analysis, we discover that the downstream model generated by synthetic data plays a crucial role in controlling the direction of gradient diversity during FL training, which enhances convergence speed and contributes to the notable accuracy boost observed with GPT-FL. Also, regardless of whether the target data falls within or outside the domain of the pre-trained generative model, GPT-FL consistently achieves significant performance gains, surpassing the results obtained by models trained solely with FL or synthetic data.
Submission history
From: Tuo Zhang [view email][v1] Sat, 3 Jun 2023 22:57:59 GMT (1662kb,D)
[v2] Wed, 5 Jul 2023 14:45:37 GMT (1662kb,D)
[v3] Fri, 29 Sep 2023 19:21:43 GMT (1249kb,D)
Link back to: arXiv, form interface, contact.