Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Low-Rank Semidefinite Programs via Bilinear Factorization
(Submitted on 3 Nov 2018 (v1), revised 1 Feb 2021 (this version, v6), latest version 28 Sep 2021 (v8))
Abstract: Many machine learning problems can be reduced to learning a low-rank positive semidefinite matrix (denoted as $Z$), which encounters semidefinite program (SDP). Existing SDP solvers are often expensive for large-scale learning. To avoid directly solving SDP, some works convert SDP into a nonconvex program by factorizing $Z$ \textit{quadraticly} as $XX^\top$. However, this would bring higher-order nonlinearity, resulting in scarcity of structure in subsequent optimization. In this paper, we propose a novel surrogate for SDP learning, in which the structure of subproblem is exploited. More specifically, we surrogate unconstrained SDP by a biconvex problem, through factorizing $Z$ \textit{bilinearly} as $XY^\top$ and using a Courant penalty to penalize the difference of $X$ and $Y$, in which the resultant subproblems in terms of $X$ and $Y$ are convex respectively. Furthermore, we provide a theoretical bound for the associated penalty parameter under the assumption that the subobjective function of $X$ or $Y$ is $L$-Lipschitz-smooth and $\sigma$-strongly convex, such that the proposed surrogate will solve the original SDP when the penalty parameter is larger than this bound (that is $\gamma>\frac{1}{4}(L-\sigma)$). Experiments on two SDP-related applications demonstrate that the proposed algorithm is as accurate as the state-of-the-art, but is faster on large-scale learning.
Submission history
From: En-Liang Hu [view email][v1] Sat, 3 Nov 2018 12:15:42 GMT (104kb,D)
[v2] Tue, 24 Dec 2019 03:32:56 GMT (107kb,D)
[v3] Wed, 25 Dec 2019 03:13:44 GMT (107kb,D)
[v4] Sat, 7 Mar 2020 11:02:05 GMT (267kb,D)
[v5] Wed, 20 Jan 2021 11:19:37 GMT (268kb,D)
[v6] Mon, 1 Feb 2021 06:03:46 GMT (0kb,I)
[v7] Tue, 2 Feb 2021 02:59:16 GMT (0kb,I)
[v8] Tue, 28 Sep 2021 13:36:31 GMT (104kb,D)
Link back to: arXiv, form interface, contact.