Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Bayesian $l_0$ Regularized Least Squares
(Submitted on 31 May 2017 (this version), latest version 18 Dec 2018 (v2))
Abstract: Bayesian $l_0$-regularized least squares provides a variable selection technique for high dimensional predictors. The challenge in $l_0$ regularization is optimizing a non-convex objective function via search over model space consisting of all possible predictor combinations, a NP-hard task. Spike-and-slab (a.k.a. Bernoulli-Gaussian, BG) priors are the gold standard for Bayesian variable selection, with a caveat of computational speed and scalability. We show that a Single Best Replacement (SBR) algorithm is a fast scalable alternative. Although SBR calculates a sparse posterior mode, we show that it possesses a number of equivalences and optimality properties of a posterior mean. To illustrate our methodology, we provide simulation evidence and a real data example on the statistical properties and computational efficiency of SBR versus direct posterior sampling using spike-and-slab priors. Finally, we conclude with directions for future research.
Submission history
From: Lei Sun [view email][v1] Wed, 31 May 2017 21:29:40 GMT (498kb)
[v2] Tue, 18 Dec 2018 17:10:53 GMT (68kb)
Link back to: arXiv, form interface, contact.