We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.ST

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Statistics Theory

Title: Robust linear least squares regression

Abstract: We consider the problem of robustly predicting as well as the best linear combination of $d$ given functions in least squares regression, and variants of this problem including constraints on the parameters of the linear combination. For the ridge estimator and the ordinary least squares estimator, and their variants, we provide new risk bounds of order $d/n$ without logarithmic factor unlike some standard results, where $n$ is the size of the training data. We also provide a new estimator with better deviations in the presence of heavy-tailed noise. It is based on truncating differences of losses in a min--max framework and satisfies a $d/n$ risk bound both in expectation and in deviations. The key common surprising factor of these results is the absence of exponential moment condition on the output distribution while achieving exponential deviations. All risk bounds are obtained through a PAC-Bayesian analysis on truncated differences of losses. Experimental results strongly back up our truncated min--max estimator.
Comments: Published in at this http URL the Annals of Statistics (this http URL) by the Institute of Mathematical Statistics (this http URL). arXiv admin note: significant text overlap with arXiv:0902.1733
Subjects: Statistics Theory (math.ST)
Journal reference: Annals of Statistics 2011, Vol. 39, No. 5, 2766-2794
DOI: 10.1214/11-AOS918
Report number: IMS-AOS-AOS918
Cite as: arXiv:1010.0074 [math.ST]
  (or arXiv:1010.0074v3 [math.ST] for this version)

Submission history

From: Jean-Yves Audibert [view email]
[v1] Fri, 1 Oct 2010 06:20:59 GMT (121kb)
[v2] Sun, 18 Sep 2011 18:06:15 GMT (133kb)
[v3] Thu, 23 Feb 2012 07:55:39 GMT (588kb)

Link back to: arXiv, form interface, contact.