Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: A First Order Free Lunch for SQRT-Lasso
(Submitted on 25 May 2016 (this version), latest version 14 Apr 2019 (v6))
Abstract: Many statistical machine learning techniques sacrifice convenient computational structures to gain estimation robustness and modeling flexibility. In this paper, we study this fundamental tradeoff through a SQRT-Lasso problem for sparse linear regression and sparse precision matrix estimation in high dimensions. We explain how novel optimization techniques help address these computational challenges. Particularly, we propose a pathwise iterative smoothing shrinkage thresholding algorithm for solving the SQRT-Lasso optimization problem. We further provide a novel model-based perspective for analyzing the smoothing optimization framework, which allows us to establish a nearly linear convergence (R-linear convergence) guarantee for our proposed algorithm. This implies that solving the SQRT-Lasso optimization is almost as easy as solving the Lasso optimization. Moreover, we show that our proposed algorithm can also be applied to sparse precision matrix estimation, and enjoys good computational properties. Numerical experiments are provided to support our theory.
Submission history
From: Haoming Jiang [view email][v1] Wed, 25 May 2016 16:08:08 GMT (1263kb,D)
[v2] Mon, 11 Jul 2016 02:10:41 GMT (1263kb,D)
[v3] Mon, 2 Jan 2017 18:32:20 GMT (1148kb,D)
[v4] Sat, 3 Feb 2018 05:50:46 GMT (1242kb,D)
[v5] Wed, 14 Feb 2018 16:06:58 GMT (1194kb,D)
[v6] Sun, 14 Apr 2019 01:00:57 GMT (4177kb,D)
Link back to: arXiv, form interface, contact.