Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Universal Bayes consistency in metric spaces
(Submitted on 24 Jun 2019 (v1), last revised 6 Jan 2021 (this version, v7))
Abstract: We extend a recently proposed 1-nearest-neighbor based multiclass learning algorithm and prove that our modification is universally strongly Bayes-consistent in all metric spaces admitting any such learner, making it an "optimistically universal" Bayes-consistent learner. This is the first learning algorithm known to enjoy this property; by comparison, the $k$-NN classifier and its variants are not generally universally Bayes-consistent, except under additional structural assumptions, such as an inner product, a norm, finite dimension, or a Besicovitch-type property. The metric spaces in which universal Bayes consistency is possible are the "essentially separable" ones -- a notion that we define, which is more general than standard separability. The existence of metric spaces that are not essentially separable is widely believed to be independent of the ZFC axioms of set theory. We prove that essential separability exactly characterizes the existence of a universal Bayes-consistent learner for the given metric space. In particular, this yields the first impossibility result for universal Bayes consistency. Taken together, our results completely characterize strong and weak universal Bayes consistency in metric spaces.
Submission history
From: Roi Weiss [view email][v1] Mon, 24 Jun 2019 11:29:00 GMT (60kb)
[v2] Wed, 26 Jun 2019 06:50:37 GMT (60kb)
[v3] Sat, 1 Aug 2020 21:16:08 GMT (189kb,D)
[v4] Tue, 4 Aug 2020 17:20:57 GMT (208kb,D)
[v5] Sun, 11 Oct 2020 16:31:23 GMT (197kb,D)
[v6] Fri, 16 Oct 2020 09:55:46 GMT (197kb,D)
[v7] Wed, 6 Jan 2021 22:39:49 GMT (197kb,D)
Link back to: arXiv, form interface, contact.