We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Algorithmic Regularization in Over-parameterized Matrix Recovery

Abstract: We study the problem of recovering a low-rank matrix $X^\star$ from linear measurements using an over-parameterized model. We parameterize the rank-$r$ matrix $X^\star$ by $UU^\top$ where $U\in \mathbb{R}^{d\times d}$ is a square matrix, whereas the number of linear measurements is much less than $d^2$. We show that with $\tilde{O}(dr^{2})$ random linear measurements, the gradient descent on the squared loss, starting from a small initialization, recovers $X^\star$ approximately in $\tilde{O}(\sqrt{r})$ iterations. The results solve the conjecture of Gunasekar et al. under the restricted isometry property, and demonstrate that the training algorithm can provide an implicit regularization for non-linear matrix factorization models.
Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Optimization and Control (math.OC); Machine Learning (stat.ML)
Cite as: arXiv:1712.09203 [cs.LG]
  (or arXiv:1712.09203v1 [cs.LG] for this version)

Submission history

From: Hongyang Zhang [view email]
[v1] Tue, 26 Dec 2017 08:04:43 GMT (397kb)
[v2] Wed, 21 Feb 2018 08:21:52 GMT (71kb)
[v3] Thu, 22 Feb 2018 03:50:08 GMT (71kb)
[v4] Mon, 19 Mar 2018 07:59:32 GMT (73kb)
[v5] Thu, 14 Feb 2019 00:24:05 GMT (109kb)

Link back to: arXiv, form interface, contact.