Current browse context:
math.OC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Optimal and Adaptive Monteiro-Svaiter Acceleration
(Submitted on 30 May 2022 (v1), last revised 28 Nov 2022 (this version, v2))
Abstract: We develop a variant of the Monteiro-Svaiter (MS) acceleration framework that removes the need to solve an expensive implicit equation at every iteration. Consequently, for any $p\ge 2$ we improve the complexity of convex optimization with Lipschitz $p$th derivative by a logarithmic factor, matching a lower bound. We also introduce an MS subproblem solver that requires no knowledge of problem parameters, and implement it as either a second- or first-order method by solving linear systems or applying MinRes, respectively. On logistic regression our method outperforms previous second-order momentum methods, but under-performs Newton's method; simply iterating our first-order adaptive subproblem solver performs comparably to L-BFGS.
Submission history
From: Yair Carmon [view email][v1] Mon, 30 May 2022 18:31:07 GMT (322kb,D)
[v2] Mon, 28 Nov 2022 21:35:34 GMT (325kb,D)
Link back to: arXiv, form interface, contact.