Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: The Physical Systems Behind Optimization Algorithms
(Submitted on 8 Dec 2016 (v1), revised 12 May 2017 (this version, v3), latest version 25 Oct 2018 (v5))
Abstract: We use differential equations based approaches to provide insights into analyzing the dynamics of popular optimization algorithms for machine learning. In particular, we study gradient descent, proximal gradient descent, coordinate gradient descent, proximal coordinate gradient, and Newton's methods as well as their Nesterov's accelerated variants in a unified framework motivated by a natural connection of optimization algorithms to physical systems. Our analysis is applicable to more general algorithms and optimization problems beyond convexity and strong convexity.
Submission history
From: Lin Yang [view email][v1] Thu, 8 Dec 2016 20:36:30 GMT (279kb,D)
[v2] Mon, 6 Mar 2017 17:44:34 GMT (285kb,D)
[v3] Fri, 12 May 2017 19:01:38 GMT (281kb,D)
[v4] Mon, 22 May 2017 18:24:39 GMT (392kb,D)
[v5] Thu, 25 Oct 2018 04:04:13 GMT (271kb,D)
Link back to: arXiv, form interface, contact.