We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Why Do Local Methods Solve Nonconvex Problems?

Authors: Tengyu Ma
Abstract: Non-convex optimization is ubiquitous in modern machine learning. Researchers devise non-convex objective functions and optimize them using off-the-shelf optimizers such as stochastic gradient descent and its variants, which leverage the local geometry and update iteratively. Even though solving non-convex functions is NP-hard in the worst case, the optimization quality in practice is often not an issue -- optimizers are largely believed to find approximate global minima. Researchers hypothesize a unified explanation for this intriguing phenomenon: most of the local minima of the practically-used objectives are approximately global minima. We rigorously formalize it for concrete instances of machine learning problems.
Comments: This is the Chapter 21 of the book "Beyond the Worst-Case Analysis of Algorithms"
Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Optimization and Control (math.OC); Machine Learning (stat.ML)
Cite as: arXiv:2103.13462 [cs.LG]
  (or arXiv:2103.13462v1 [cs.LG] for this version)

Submission history

From: Tengyu Ma [view email]
[v1] Wed, 24 Mar 2021 19:34:11 GMT (302kb,D)

Link back to: arXiv, form interface, contact.