Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Agnostic Learning of General ReLU Activation Using Gradient Descent
(Submitted on 4 Aug 2022)
Abstract: We provide a convergence analysis of gradient descent for the problem of agnostically learning a single ReLU function under Gaussian distributions. Unlike prior work that studies the setting of zero bias, we consider the more challenging scenario when the bias of the ReLU function is non-zero. Our main result establishes that starting from random initialization, in a polynomial number of iterations gradient descent outputs, with high probability, a ReLU function that achieves a competitive error guarantee when compared to the error of the best ReLU function. We also provide finite sample guarantees, and these techniques generalize to a broader class of marginal distributions beyond Gaussians.
Submission history
From: Aravindan Vijayaraghavan [view email][v1] Thu, 4 Aug 2022 15:14:05 GMT (66kb)
Link back to: arXiv, form interface, contact.