We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Learning the Hypotheses Space from data: Learning Space and U-curve Property

Abstract: This paper presents an extension of the classical agnostic PAC learning model in which learning problems are modelled not only by a Hypothesis Space $\mathcal{H}$, but also by a Learning Space $\mathbb{L}(\mathcal{H})$, which is a cover of $\mathcal{H}$, constrained by a VC-dimension property, that is a suitable domain for Model Selection algorithms. Our main contribution is a data driven general learning algorithm to perform regularized Model Selection on $\mathbb{L}(\mathcal{H})$. A remarkable, formally proved, consequence of this approach are conditions on $\mathbb{L}(\mathcal{H})$ and on the loss function that lead to estimated out-of-sample error surfaces which are true U-curves on $\mathbb{L}(\mathcal{H})$ chains, enabling a more efficient search on $\mathbb{L}(\mathcal{H})$. To our knowledge, this is the first rigorous result asserting that a non exhaustive search of a family of candidate models can return an optimal solution. In this new framework, an U-curve optimization algorithm becomes a natural component of Model Selection, hence of learning algorithms. The abstract general framework proposed here may have important implications on modern learning models and on areas such as Neural Architecture Search.
Comments: This paper has been withdrawn by the authors. This paper has been superseded by arXiv:2109.03866 (merged from arXiv:2001.09532 and arXiv:2001.11578)
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:2001.09532 [stat.ML]
  (or arXiv:2001.09532v3 [stat.ML] for this version)

Submission history

From: Diego Marcondes [view email]
[v1] Sun, 26 Jan 2020 22:29:33 GMT (230kb,D)
[v2] Mon, 30 Nov 2020 13:55:04 GMT (737kb,D)
[v3] Fri, 10 Sep 2021 12:33:44 GMT (0kb,I)

Link back to: arXiv, form interface, contact.