References & Citations
Statistics > Machine Learning
Title: Sparse Generalized Eigenvalue Problem: Optimal Statistical Rates via Truncated Rayleigh Flow
(Submitted on 29 Apr 2016 (this version), latest version 31 Aug 2018 (v3))
Abstract: Sparse generalized eigenvalue problem plays a pivotal role in a large family of high-dimensional learning tasks, including sparse Fisher's discriminant analysis, canonical correlation analysis, and sufficient dimension reduction. However, the theory of sparse generalized eigenvalue problem remains largely unexplored. In this paper, we exploit a non-convex optimization perspective to study this problem. In particular, we propose the truncated Rayleigh flow method (Rifle) to estimate the leading generalized eigenvector and show that it converges linearly to a solution with the optimal statistical rate of convergence. Our theory involves two key ingredients: (i) a new analysis of the gradient descent method on non-convex objective functions, as well as (ii) a fine-grained characterization of the evolution of sparsity patterns along the solution path. Thorough numerical studies are provided to back up our theory. Finally, we apply our proposed method in the context of sparse sufficient dimension reduction to two gene expression data sets.
Submission history
From: Kean Ming Tan [view email][v1] Fri, 29 Apr 2016 06:12:19 GMT (74kb,D)
[v2] Fri, 17 Aug 2018 21:48:29 GMT (71kb,D)
[v3] Fri, 31 Aug 2018 19:48:51 GMT (64kb,D)
Link back to: arXiv, form interface, contact.