Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: A Robust Accelerated Optimization Algorithm for Strongly Convex Functions
(Submitted on 13 Oct 2017 (v1), last revised 26 Feb 2018 (this version, v2))
Abstract: This work proposes an accelerated first-order algorithm we call the Robust Momentum Method for optimizing smooth strongly convex functions. The algorithm has a single scalar parameter that can be tuned to trade off robustness to gradient noise versus worst-case convergence rate. At one extreme, the algorithm is faster than Nesterov's Fast Gradient Method by a constant factor but more fragile to noise. At the other extreme, the algorithm reduces to the Gradient Method and is very robust to noise. The algorithm design technique is inspired by methods from classical control theory and the resulting algorithm has a simple analytical form. Algorithm performance is verified on a series of numerical simulations in both noise-free and relative gradient noise cases.
Submission history
From: Laurent Lessard [view email][v1] Fri, 13 Oct 2017 00:04:56 GMT (384kb,D)
[v2] Mon, 26 Feb 2018 16:16:12 GMT (384kb,D)
Link back to: arXiv, form interface, contact.