Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: ReLU Regression with Massart Noise
(Submitted on 10 Sep 2021 (v1), last revised 26 Jan 2022 (this version, v2))
Abstract: We study the fundamental problem of ReLU regression, where the goal is to fit Rectified Linear Units (ReLUs) to data. This supervised learning task is efficiently solvable in the realizable setting, but is known to be computationally hard with adversarial label noise. In this work, we focus on ReLU regression in the Massart noise model, a natural and well-studied semi-random noise model. In this model, the label of every point is generated according to a function in the class, but an adversary is allowed to change this value arbitrarily with some probability, which is {\em at most} $\eta < 1/2$. We develop an efficient algorithm that achieves exact parameter recovery in this model under mild anti-concentration assumptions on the underlying distribution. Such assumptions are necessary for exact recovery to be information-theoretically possible. We demonstrate that our algorithm significantly outperforms naive applications of $\ell_1$ and $\ell_2$ regression on both synthetic and real data.
Submission history
From: Jongho Park [view email][v1] Fri, 10 Sep 2021 02:13:22 GMT (128kb,D)
[v2] Wed, 26 Jan 2022 01:58:30 GMT (439kb,D)
Link back to: arXiv, form interface, contact.