We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.NA

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Mathematics > Optimization and Control

Title: Negative curvature obstructs acceleration for strongly geodesically convex optimization, even with exact first-order oracles

Abstract: Hamilton and Moitra (2021) showed that, in certain regimes, it is not possible to accelerate Riemannian gradient descent in the hyperbolic plane if we restrict ourselves to algorithms which make queries in a (large) bounded domain and which receive gradients and function values corrupted by a (small) amount of noise. We show that acceleration remains unachievable for any deterministic algorithm which receives exact gradient and function-value information (unbounded queries, no noise). Our results hold for the classes of strongly and nonstrongly geodesically convex functions, and for a large class of Hadamard manifolds including hyperbolic spaces and the symmetric space $\mathrm{SL}(n) / \mathrm{SO}(n)$ of positive definite $n \times n$ matrices of determinant one. This cements a surprising gap between the complexity of convex optimization and geodesically convex optimization: for hyperbolic spaces, Riemannian gradient descent is optimal on the class of smooth and and strongly geodesically convex functions, in the regime where the condition number scales with the radius of the optimization domain. The key idea for proving the lower bound consists of perturbing the hard functions of Hamilton and Moitra (2021) with sums of bump functions chosen by a resisting oracle.
Comments: v2 to v3: Updated and shortened to reflect COLT 2022 version. Results on nonstrongly g-convex case (former Sec. 5) and reduction to Euclidean convexity (former Sec. 6) are now in Sec. 3 and App. D of "Curvature and Complexity: Better lower bounds for geodesically convex optimization", COLT 2023 (arxiv.org/abs/2306.02959). v3 to v4: Added word "strongly" to title to match COLT 2022 version; Proceedings of Thirty Fifth Conference on Learning Theory, PMLR 178:496-542, 2022, this https URL
Subjects: Optimization and Control (math.OC); Computational Complexity (cs.CC); Differential Geometry (math.DG); Numerical Analysis (math.NA)
Cite as: arXiv:2111.13263 [math.OC]
  (or arXiv:2111.13263v4 [math.OC] for this version)

Submission history

From: Christopher Criscitiello [view email]
[v1] Thu, 25 Nov 2021 21:54:52 GMT (297kb,D)
[v2] Fri, 14 Jan 2022 00:54:57 GMT (296kb,D)
[v3] Tue, 6 Jun 2023 06:12:04 GMT (328kb,D)
[v4] Thu, 8 Jun 2023 18:29:10 GMT (328kb,D)

Link back to: arXiv, form interface, contact.