We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ME

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Methodology

Title: Hierarchical Sparse Modeling: A Choice of Two Group Lasso Formulations

Abstract: Demanding sparsity in estimated models has become a routine practice in statistics. In many situations, we wish to require that the sparsity patterns attained honor certain problem-specific constraints. Hierarchical sparse modeling (HSM) refers to situations in which these constraints specify that one set of parameters be set to zero whenever another is set to zero. In recent years, numerous papers have developed convex regularizers for this form of sparsity structure, which arises in many areas of statistics including interaction modeling, time series analysis, and covariance estimation. In this paper, we observe that these methods fall into two frameworks, the group lasso (GL) and latent overlapping group lasso (LOG), which have not been systematically compared in the context of HSM. The purpose of this paper is to provide a side-by-side comparison of these two frameworks for HSM in terms of their statistical properties and computational efficiency. We call special attention to GL's more aggressive shrinkage of parameters deep in the hierarchy, a property not shared by LOG. In terms of computation, we introduce a finite-step algorithm that exactly solves the proximal operator of LOG for a certain simple HSM structure; we later exploit this to develop a novel path-based block coordinate descent scheme for general HSM structures. Both algorithms greatly improve the computational performance of LOG. Finally, we compare the two methods in the context of covariance estimation, where we introduce a new sparsely-banded estimator using LOG, which we show achieves the statistical advantages of an existing GL-based method but is simpler to express and more efficient to compute.
Comments: 30 pages, 13 figures
Subjects: Methodology (stat.ME); Statistics Theory (math.ST); Computation (stat.CO); Machine Learning (stat.ML)
Journal reference: Statist. Sci. 32 (2017), no. 4, 531--560
DOI: 10.1214/17-STS622
Cite as: arXiv:1512.01631 [stat.ME]
  (or arXiv:1512.01631v4 [stat.ME] for this version)

Submission history

From: Xiaohan Yan [view email]
[v1] Sat, 5 Dec 2015 07:00:54 GMT (1548kb,D)
[v2] Tue, 29 Nov 2016 02:13:49 GMT (7425kb,D)
[v3] Mon, 3 Jul 2017 04:03:45 GMT (4015kb,D)
[v4] Wed, 29 Nov 2017 20:05:56 GMT (4133kb)

Link back to: arXiv, form interface, contact.