Current browse context:
stat.ME
Change to browse by:
References & Citations
Statistics > Methodology
Title: Feature Selection in High-dimensional Space Using Graph-Based Methods
(Submitted on 28 Aug 2021 (v1), last revised 27 Jan 2023 (this version, v2))
Abstract: High-dimensional feature selection is a central problem in a variety of application domains such as machine learning, image analysis, and genomics. In this paper, we propose graph-based tests as a useful basis for feature selection. We describe an algorithm for selecting informative features in high-dimensional data, where each observation comes from one of $K$ different distributions. Our algorithm can be applied in a completely nonparametric setup without any distributional assumptions on the data, and it aims at outputting those features in the data, that contribute the most to the overall distributional variation. At the heart of our method is the recursive application of distribution-free graph-based tests on subsets of the feature set, located at different depths of a hierarchical clustering tree constructed from the data. Our algorithm recovers all truly contributing features with high probability, while ensuring optimal control on false-discovery. Finally, we show the superior performance of our method over other existing ones through synthetic data, and also demonstrate the utility of the method on two real-life datasets from the domains of climate change and single cell transcriptomics.
Submission history
From: Somabha Mukherjee [view email][v1] Sat, 28 Aug 2021 17:50:43 GMT (171kb,D)
[v2] Fri, 27 Jan 2023 02:02:12 GMT (310kb,D)
Link back to: arXiv, form interface, contact.