References & Citations
Mathematics > Numerical Analysis
Title: Practical Leverage-Based Sampling for Low-Rank Tensor Decomposition
(Submitted on 30 Jun 2020 (v1), revised 10 Dec 2020 (this version, v2), latest version 3 Jan 2022 (v3))
Abstract: The low-rank canonical polyadic tensor decomposition is useful in data analysis and can be computed by solving a sequence of overdetermined least squares subproblems. Motivated by consideration of sparse tensors, we propose sketching each subproblem using leverage scores to select a subset of the rows, with probabilistic guarantees of the solution accuracy. We randomly sample rows proportional to leverage score upper bounds that can be efficiently computed using the special Khatri-Rao subproblem structure inherent in tensor decomposition. Crucially, the number of rows in the sketched system is independent of the number of nonzeros in the full tensor and the number of rows in the full system. Along the way, we provide a practical solution to the generic matrix sketching problem of sampling overabundance for high-leverage-score rows, proposing to include such rows deterministically and combine repeated samples in the sketched system; we conjecture that this can lead to improved theoretical bounds. Numerical results on real-world large-scale tensors show the method is significantly faster than deterministic methods without sacrificing accuracy.
Submission history
From: Tamara Kolda [view email][v1] Tue, 30 Jun 2020 00:01:22 GMT (1740kb,D)
[v2] Thu, 10 Dec 2020 00:18:58 GMT (10963kb,D)
[v3] Mon, 3 Jan 2022 21:08:52 GMT (11501kb,AD)
Link back to: arXiv, form interface, contact.