We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Feature Selection using e-values

Abstract: In the context of supervised parametric models, we introduce the concept of e-values. An e-value is a scalar quantity that represents the proximity of the sampling distribution of parameter estimates in a model trained on a subset of features to that of the model trained on all features (i.e. the full model). Under general conditions, a rank ordering of e-values separates models that contain all essential features from those that do not.
The e-values are applicable to a wide range of parametric models. We use data depths and a fast resampling-based algorithm to implement a feature selection procedure using e-values, providing consistency results. For a $p$-dimensional feature space, this procedure requires fitting only the full model and evaluating $p+1$ models, as opposed to the traditional requirement of fitting and evaluating $2^p$ models. Through experiments across several model settings and synthetic and real datasets, we establish that the e-values method as a promising general alternative to existing model-specific methods of feature selection.
Comments: accepted in ICML-2022
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Methodology (stat.ME)
Journal reference: Proceedings of the 39th International Conference on Machine Learning, PMLR 162:14753-14773, 2022, https://proceedings.mlr.press/v162/majumdar22a.html
Cite as: arXiv:2206.05391 [stat.ML]
  (or arXiv:2206.05391v2 [stat.ML] for this version)

Submission history

From: Subhabrata Majumdar [view email]
[v1] Sat, 11 Jun 2022 01:34:29 GMT (413kb,D)
[v2] Thu, 16 Jun 2022 16:00:44 GMT (405kb,D)

Link back to: arXiv, form interface, contact.