We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Bayesian $l_0$-regularized Least Squares

Abstract: Bayesian $l_0$-regularized least squares is a variable selection technique for high dimensional predictors. The challenge is optimizing a non-convex objective function via search over model space consisting of all possible predictor combinations. Spike-and-slab (a.k.a. Bernoulli-Gaussian) priors are the gold standard for Bayesian variable selection, with a caveat of computational speed and scalability. Single Best Replacement (SBR) provides a fast scalable alternative. We provide a link between Bayesian regularization and proximal updating, which provides an equivalence between finding a posterior mode and a posterior mean with a different regularization prior. This allows us to use SBR to find the spike-and-slab estimator. To illustrate our methodology, we provide simulation evidence and a real data example on the statistical properties and computational efficiency of SBR versus direct posterior sampling using spike-and-slab priors. Finally, we conclude with directions for future research.
Comments: 22 pages, 6 figures, 1 table
Subjects: Machine Learning (stat.ML); Computation (stat.CO)
MSC classes: 62-04
DOI: 10.1002/asmb.2381
Cite as: arXiv:1706.00098 [stat.ML]
  (or arXiv:1706.00098v2 [stat.ML] for this version)

Submission history

From: Lei Sun [view email]
[v1] Wed, 31 May 2017 21:29:40 GMT (498kb)
[v2] Tue, 18 Dec 2018 17:10:53 GMT (68kb)

Link back to: arXiv, form interface, contact.