Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Out-of-distribution Detection Using Kernel Density Polytopes
(Submitted on 31 Jan 2022 (this version), latest version 12 Mar 2024 (v7))
Abstract: Any reasonable machine learning (ML) model should not only interpolate efficiently in between the training samples provided (in-distribution region), but also approach the extrapolative or out-of-distribution (OOD) region without being overconfident. Our experiment on human subjects justifies the aforementioned properties for human intelligence as well. Many state-of-the-art algorithms have tried to fix the overconfidence problem of ML models in the OOD region. However, in doing so, they have often impaired the in-distribution performance of the model. Our key insight is that ML models partition the feature space into polytopes and learn constant (random forests) or affine (ReLU networks) functions over those polytopes. This leads to the OOD overconfidence problem for the polytopes which lie in the training data boundary and extend to infinity. To resolve this issue, we propose kernel density methods that fit Gaussian kernel over the polytopes, which are learned using ML models. Specifically, we introduce two variants of kernel density polytopes: Kernel Density Forest (KDF) and Kernel Density Network (KDN) based on random forests and deep networks, respectively. Studies on various simulation settings show that both KDF and KDN achieve uniform confidence over the classes in the OOD region while maintaining good in-distribution accuracy compared to that of their respective parent models.
Submission history
From: Jayanta Dey [view email][v1] Mon, 31 Jan 2022 05:07:16 GMT (16278kb,D)
[v2] Sun, 6 Feb 2022 14:38:23 GMT (16262kb,D)
[v3] Mon, 14 Feb 2022 14:14:47 GMT (16263kb,D)
[v4] Fri, 25 Mar 2022 03:41:23 GMT (16263kb,D)
[v5] Fri, 19 May 2023 21:15:28 GMT (18650kb,D)
[v6] Thu, 19 Oct 2023 02:27:07 GMT (33996kb,D)
[v7] Tue, 12 Mar 2024 12:57:20 GMT (33151kb,D)
Link back to: arXiv, form interface, contact.