We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: On the Equivalence between Online and Private Learnability beyond Binary Classification

Abstract: Alon et al. [2019] and Bun et al. [2020] recently showed that online learnability and private PAC learnability are equivalent in binary classification. We investigate whether this equivalence extends to multi-class classification and regression. First, we show that private learnability implies online learnability in both settings. Our extension involves studying a novel variant of the Littlestone dimension that depends on a tolerance parameter and on an appropriate generalization of the concept of threshold functions beyond binary classification. Second, we show that while online learnability continues to imply private learnability in multi-class classification, current proof techniques encounter significant hurdles in the regression setting. While the equivalence for regression remains open, we provide non-trivial sufficient conditions for an online learnable class to also be privately learnable.
Comments: An earlier version of this manuscript claimed an upper bound over the sample complexity that is exponential in the Littlestone dimension. The argument contained a technical mistake, and the current version presents a correction that deteriorates the dependence on the Littlestone dimension from exponential to doubly exponential. arXiv admin note: text overlap with arXiv:2003.00563 by other authors
Subjects: Machine Learning (stat.ML); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Cite as: arXiv:2006.01980 [stat.ML]
  (or arXiv:2006.01980v3 [stat.ML] for this version)

Submission history

From: Baekjin Kim [view email]
[v1] Tue, 2 Jun 2020 23:30:41 GMT (282kb,D)
[v2] Sun, 1 Nov 2020 01:25:59 GMT (36kb,D)
[v3] Sat, 9 Oct 2021 03:24:30 GMT (318kb,D)

Link back to: arXiv, form interface, contact.