Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Provably Useful Kernel Matrix Approximation in Linear Time
(Submitted on 24 May 2016 (v1), revised 31 May 2016 (this version, v2), latest version 3 Nov 2017 (v5))
Abstract: We give the first algorithms for kernel matrix approximation that run in time linear in the number of data points and output an approximation which gives provable guarantees when used in many downstream learning tasks, including kernel principal component analysis, kernel $k$-means clustering, kernel ridge regression, and kernel canonical correlation analysis.
Our methods require just $\tilde O(n\cdot k)$ kernel evaluations and $\tilde O(n \cdot k^2)$ additional runtime, where $n$ is the number of training data points and $k$ is a target rank or effective dimensionality parameter. These runtimes are significantly sub-linear in the size of the $n \times n$ kernel matrix and apply to any kernel matrix, without assuming regularity or incoherence conditions.
The algorithms are based on a ridge leverage score Nystr\"om sampling scheme (RLS-Nystr\"om) which was recently shown to yield strong kernel approximations, but which had no efficient implementation. We address this shortcoming by introducing fast recursive sampling methods for RLS-Nystr\"om, while at the same time proving extended approximation guarantees for this promising new method.
Submission history
From: Cameron Musco [view email][v1] Tue, 24 May 2016 18:56:57 GMT (31kb)
[v2] Tue, 31 May 2016 19:48:44 GMT (32kb)
[v3] Tue, 28 Feb 2017 16:37:17 GMT (1625kb)
[v4] Thu, 16 Mar 2017 17:58:14 GMT (1627kb)
[v5] Fri, 3 Nov 2017 14:40:15 GMT (1630kb)
Link back to: arXiv, form interface, contact.