We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Fast rates in structured prediction

Abstract: Discrete supervised learning problems such as classification are often tackled by introducing a continuous surrogate problem akin to regression. Bounding the original error, between estimate and solution, by the surrogate error endows discrete problems with convergence rates already shown for continuous instances. Yet, current approaches do not leverage the fact that discrete problems are essentially predicting a discrete output when continuous problems are predicting a continuous value. In this paper, we tackle this issue for general structured prediction problems, opening the way to "super fast" rates, that is, convergence rates for the excess risk faster than $n^{-1}$, where $n$ is the number of observations, with even exponential rates with the strongest assumptions. We first illustrate it for predictors based on nearest neighbors, generalizing rates known for binary classification to any discrete problem within the framework of structured prediction. We then consider kernel ridge regression where we improve known rates in $n^{-1/4}$ to arbitrarily fast rates, depending on a parameter characterizing the hardness of the problem, thus allowing, under smoothness assumptions, to bypass the curse of dimensionality.
Comments: 14 main pages, 3 main figures, 43 pages, 4 figures (with appendix)
Subjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Statistics Theory (math.ST)
MSC classes: 68T05
ACM classes: I.2.6; F.2.2; G.3
Journal reference: Conference on Learning Theory, PMLR 134, 2021
Cite as: arXiv:2102.00760 [stat.ML]
  (or arXiv:2102.00760v3 [stat.ML] for this version)

Submission history

From: Vivien Cabannes [view email]
[v1] Mon, 1 Feb 2021 10:50:04 GMT (61kb,D)
[v2] Tue, 8 Jun 2021 13:02:31 GMT (187kb)
[v3] Thu, 15 Jul 2021 15:04:41 GMT (204kb)

Link back to: arXiv, form interface, contact.