We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Minimum discrepancy principle strategy for choosing $k$ in $k$-NN regression

Abstract: This paper presents a novel data-driven strategy to choose the hyperparameter $k$ in the $k$-NN regression estimator. We treat the problem of choosing the hyperparameter as an iterative procedure (over $k$) and propose using an easily implemented in practice strategy based on the idea of early stopping and the minimum discrepancy principle. This estimation strategy is proven to be minimax optimal, under the fixed-design assumption on covariates, over different smoothness function classes, for instance, the Lipschitz functions class on a bounded domain. After that, the novel strategy shows consistent simulations results on artificial and real-world data sets in comparison to other model selection strategies such as the Hold-out method.
Comments: 38 pages
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
Cite as: arXiv:2008.08718 [stat.ML]
  (or arXiv:2008.08718v1 [stat.ML] for this version)

Submission history

From: Yaroslav Averyanov [view email]
[v1] Thu, 20 Aug 2020 00:13:19 GMT (93kb,D)
[v2] Wed, 20 Jan 2021 15:11:35 GMT (178kb)
[v3] Fri, 26 Feb 2021 11:53:33 GMT (212kb)
[v4] Wed, 5 May 2021 11:33:16 GMT (68kb)

Link back to: arXiv, form interface, contact.