Current browse context:
cs.CR
Change to browse by:
References & Citations
Computer Science > Cryptography and Security
Title: Minimal Model Structure Analysis for Input Reconstruction in Federated Learning
(Submitted on 29 Oct 2020 (v1), last revised 5 Nov 2021 (this version, v4))
Abstract: \ac{fl} proposed a distributed \ac{ml} framework where every distributed worker owns a complete copy of global model and their own data. The training is occurred locally, which assures no direct transmission of training data. However, the recent work \citep{zhu2019deep} demonstrated that input data from a neural network may be reconstructed only using knowledge of gradients of that network, which completely breached the promise of \ac{fl} and sabotaged the user privacy.
In this work, we aim to further explore the theoretical limits of reconstruction, speedup and stabilize the reconstruction procedure. We show that a single input may be reconstructed with the analytical form, regardless of network depth using a fully-connected neural network with one hidden node. Then we generalize this result to a gradient averaged over batches of size $B$. In this case, the full batch can be reconstructed if the number of hidden units exceeds $B$. For a \ac{cnn}, the number of required kernels in convolutional layers is decided by multiple factors, e.g., padding, kernel and stride size, etc. We require the number of kernels $h\geq (\frac{d}{d^{\prime}})^2C$, where we define $d$ as input width, $d^{\prime}$ as output width after convolutional layer, and $C$ as channel number of input. We validate our observation and demonstrate the improvements using bio-medical (fMRI, \ac{wbc}) and benchmark data (MNIST, Kuzushiji-MNIST, CIFAR100, ImageNet and face images).
Submission history
From: Jia Qian [view email][v1] Thu, 29 Oct 2020 16:05:45 GMT (1897kb,D)
[v2] Thu, 25 Feb 2021 15:50:14 GMT (6397kb,D)
[v3] Thu, 25 Mar 2021 10:03:16 GMT (7665kb,D)
[v4] Fri, 5 Nov 2021 10:48:04 GMT (3839kb,D)
Link back to: arXiv, form interface, contact.