Current browse context:
cs.CC
Change to browse by:
References & Citations
Mathematics > Optimization and Control
Title: Complexity-optimal and Parameter-free First-order Methods for Finding Stationary Points of Composite Optimization Problems
(Submitted on 25 May 2022 (v1), last revised 16 Feb 2024 (this version, v4))
Abstract: This paper develops and analyzes an accelerated proximal descent method for finding stationary points of nonconvex composite optimization problems. The objective function is of the form $f+h$ where $h$ is a proper closed convex function, $f$ is a differentiable function on the domain of $h$, and $\nabla f$ is Lipschitz continuous on the domain of $h$. The main advantage of this method is that it is "parameter-free" in the sense that it does not require knowledge of the Lipschitz constant of $\nabla f$ or of any global topological properties of $f$. It is shown that the proposed method can obtain an $\varepsilon$-approximate stationary point with iteration complexity bounds that are optimal, up to logarithmic terms over $\varepsilon$, in both the convex and nonconvex settings. Some discussion is also given about how the proposed method can be leveraged in other existing optimization frameworks, such as min-max smoothing and penalty frameworks for constrained programming, to create more specialized parameter-free methods. Finally, numerical experiments are presented to support the practical viability of the method.
Submission history
From: Weiwei Kong [view email][v1] Wed, 25 May 2022 21:29:03 GMT (51kb,D)
[v2] Sat, 11 Feb 2023 21:00:54 GMT (3256kb,D)
[v3] Sun, 17 Sep 2023 19:27:14 GMT (2650kb,D)
[v4] Fri, 16 Feb 2024 02:38:58 GMT (2941kb,D)
Link back to: arXiv, form interface, contact.