Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Private Topic Modeling
(Submitted on 14 Sep 2016 (v1), last revised 3 Dec 2018 (this version, v3))
Abstract: We develop a privatised stochastic variational inference method for Latent Dirichlet Allocation (LDA). The iterative nature of stochastic variational inference presents challenges: multiple iterations are required to obtain accurate posterior distributions, yet each iteration increases the amount of noise that must be added to achieve a reasonable degree of privacy. We propose a practical algorithm that overcomes this challenge by combining: (1) an improved composition method for differential privacy, called the moments accountant, which provides a tight bound on the privacy cost of multiple variational inference iterations and thus significantly decreases the amount of additive noise; and (2) privacy amplification resulting from subsampling of large-scale data. Focusing on conjugate exponential family models, in our private variational inference, all the posterior distributions will be privatised by simply perturbing expected sufficient statistics. Using Wikipedia data, we illustrate the effectiveness of our algorithm for large-scale data.
Submission history
From: Mijung Park [view email][v1] Wed, 14 Sep 2016 03:18:36 GMT (5348kb,D)
[v2] Mon, 28 Nov 2016 20:56:45 GMT (2287kb,D)
[v3] Mon, 3 Dec 2018 19:58:47 GMT (39kb,D)
Link back to: arXiv, form interface, contact.