We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: On Deep Learning with Label Differential Privacy

Abstract: In many machine learning applications, the training data can contain highly sensitive personal information. Training large-scale deep models that are guaranteed not to leak sensitive information while not compromising their accuracy has been a significant challenge. In this work, we study the multi-class classification setting where the labels are considered sensitive and ought to be protected. We propose a new algorithm for training deep neural networks with label differential privacy, and run evaluations on several datasets. For Fashion MNIST and CIFAR-10, we demonstrate that our algorithm achieves significantly higher accuracy than the state-of-the-art, and in some regimes comes close to the non-private baselines. We also provide non-trivial training results for the the challenging CIFAR-100 dataset. We complement our algorithm with theoretical findings showing that in the setting of convex empirical risk minimization, the sample complexity of training with label differential privacy is dimension-independent, which is in contrast to vanilla differential privacy.
Comments: 26 pages, 4 figures
Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS)
Cite as: arXiv:2102.06062 [cs.LG]
  (or arXiv:2102.06062v1 [cs.LG] for this version)

Submission history

From: Chiyuan Zhang [view email]
[v1] Thu, 11 Feb 2021 15:09:06 GMT (93kb,D)
[v2] Tue, 26 Oct 2021 20:44:02 GMT (145kb,D)

Link back to: arXiv, form interface, contact.