Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent
(Submitted on 6 Jun 2022 (v1), last revised 2 Sep 2023 (this version, v6))
Abstract: Differentially private stochastic gradient descent (DP-SGD) is the workhorse algorithm for recent advances in private deep learning. It provides a single privacy guarantee to all datapoints in the dataset. We propose output-specific $(\varepsilon,\delta)$-DP to characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We also design an efficient algorithm to investigate individual privacy across a number of datasets. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. We further discover that the training loss and the privacy parameter of an example are well-correlated. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees. For example, on CIFAR-10, the average $\varepsilon$ of the class with the lowest test accuracy is 44.2\% higher than that of the class with the highest accuracy.
Submission history
From: Da Yu [view email][v1] Mon, 6 Jun 2022 13:49:37 GMT (451kb,D)
[v2] Tue, 7 Jun 2022 01:46:04 GMT (451kb,D)
[v3] Thu, 28 Jul 2022 14:57:15 GMT (2397kb,D)
[v4] Fri, 29 Jul 2022 03:31:22 GMT (2395kb,D)
[v5] Sun, 5 Feb 2023 13:43:30 GMT (3050kb,D)
[v6] Sat, 2 Sep 2023 21:57:30 GMT (4541kb,D)
Link back to: arXiv, form interface, contact.