Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Differentially Private Variational Dropout
(Submitted on 30 Nov 2017 (this version), latest version 16 Dec 2017 (v3))
Abstract: Deep neural networks with their large number of parameters are highly flexible learning systems. The high flexibility in such networks brings with some serious problems such as overfitting, and regularization is used to address this problem. A currently popular and effective regularization technique for controlling the overfitting is dropout. Often, large data collections required for neural networks contain sensitive information such as the medical histories of patients, and the privacy of the training data should be protected. In this paper, we modify the recently proposed variational dropout technique which provided an elegant Bayesian interpretation to dropout, and show that the intrinsic noise in the variational dropout can be exploited to obtain a degree of differential privacy. The iterative nature of training neural networks presents a challenge for privacy-preserving estimation since multiple iterations increase the amount of noise added. We overcome this by using a relaxed notion of differential privacy, called concentrated differential privacy, which provides tighter estimates on the overall privacy loss. We demonstrate the accuracy of our privacy-preserving variational dropout algorithm on benchmark datasets.
Submission history
From: Beyza Ermis Ms [view email][v1] Thu, 30 Nov 2017 21:32:27 GMT (34kb)
[v2] Tue, 12 Dec 2017 20:05:27 GMT (0kb,I)
[v3] Sat, 16 Dec 2017 10:41:02 GMT (47kb)
Link back to: arXiv, form interface, contact.