Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Biconvex Landscape In SDP-Related Learning
(Submitted on 3 Nov 2018 (v1), revised 24 Dec 2019 (this version, v2), latest version 28 Sep 2021 (v8))
Abstract: Many machine learning problems can be reduced to learning a low-rank positive semidefinite matrix (denoted as $Z$), which encounters semidefinite program (SDP). Existing SDP solvers are often expensive for large-scale learning. To avoid directly solving SDP, some works convert SDP into a nonconvex program by factorizing $Z$ as $XX^\top$. However, this would bring higher-order nonlinearity, resulting in scarcity of structure in subsequent optimization. In this paper, we propose a novel surrogate for SDP-related learning, in which the structure of subproblem is exploited. More specifically, we surrogate unconstrained SDP by a biconvex problem, through factorizing $Z$ as $XY^\top$ and using a Courant penalty to penalize the difference of $X$ and $Y$, in which the resultant subproblems are convex. Furthermore, we provide a theoretical bound for the associated penalty parameter under the assumption that the objective function is Lipschitz-smooth and strongly convex, such that the proposed surrogate will solve the original SDP when the penalty parameter is larger than this bound. Experiments on two SDP-related machine learning applications demonstrate that the proposed algorithm is as accurate as the state-of-the-art, but is faster on large-scale learning.
Submission history
From: En-Liang Hu [view email][v1] Sat, 3 Nov 2018 12:15:42 GMT (104kb,D)
[v2] Tue, 24 Dec 2019 03:32:56 GMT (107kb,D)
[v3] Wed, 25 Dec 2019 03:13:44 GMT (107kb,D)
[v4] Sat, 7 Mar 2020 11:02:05 GMT (267kb,D)
[v5] Wed, 20 Jan 2021 11:19:37 GMT (268kb,D)
[v6] Mon, 1 Feb 2021 06:03:46 GMT (0kb,I)
[v7] Tue, 2 Feb 2021 02:59:16 GMT (0kb,I)
[v8] Tue, 28 Sep 2021 13:36:31 GMT (104kb,D)
Link back to: arXiv, form interface, contact.