We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Attribute-Efficient Learning of Halfspaces with Malicious Noise: Near-Optimal Label Complexity and Noise Tolerance

Abstract: This paper is concerned with computationally efficient learning of homogeneous sparse halfspaces in $\mathbb{R}^d$ under noise. Though recent works have established attribute-efficient learning algorithms under various types of label noise (e.g. bounded noise), it remains an open question when and how $s$-sparse halfspaces can be efficiently learned under the challenging malicious noise model, where an adversary may corrupt both the unlabeled examples and the labels. We answer this question in the affirmative by designing a computationally efficient active learning algorithm with near-optimal label complexity of $\tilde{O}\big({s \log^4 \frac d \epsilon} \big)$ and noise tolerance $\eta = \Omega(\epsilon)$, where $\epsilon \in (0, 1)$ is the target error rate, under the assumption that the distribution over (uncorrupted) unlabeled examples is isotropic log-concave. Our algorithm can be straightforwardly tailored to the passive learning setting, and we show that the sample complexity is $\tilde{O}\big({\frac 1 \epsilon s^2 \log^5 d} \big)$ which also enjoys the attribute efficiency. Our main techniques include attribute-efficient paradigms for instance reweighting and for empirical risk minimization, and a new analysis of uniform concentration for unbounded data -- all of them crucially take the structure of the underlying halfspace into account.
Comments: V1/V2 had a problematic argument on polynomial-time solvability of a form of sparse principal component analysis. V3 fixed it by using a new approach based on semidefinite programming. V4/V5 polishes the writing and is accepted to ALT 2021
Subjects: Machine Learning (stat.ML); Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG)
Cite as: arXiv:2006.03781 [stat.ML]
  (or arXiv:2006.03781v5 [stat.ML] for this version)

Submission history

From: Jie Shen [view email]
[v1] Sat, 6 Jun 2020 04:57:39 GMT (39kb)
[v2] Sat, 25 Jul 2020 01:57:38 GMT (69kb)
[v3] Tue, 11 Aug 2020 07:26:35 GMT (56kb)
[v4] Wed, 23 Dec 2020 05:47:58 GMT (54kb)
[v5] Tue, 2 Mar 2021 07:19:55 GMT (79kb)

Link back to: arXiv, form interface, contact.