We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Adaptive first-order methods revisited: Convex optimization without Lipschitz requirements

Abstract: We propose a new family of adaptive first-order methods for a class of convex minimization problems that may fail to be Lipschitz continuous or smooth in the standard sense. Specifically, motivated by a recent flurry of activity on non-Lipschitz (NoLips) optimization, we consider problems that are continuous or smooth relative to a reference Bregman function - as opposed to a global, ambient norm (Euclidean or otherwise). These conditions encompass a wide range of problems with singular objectives, such as Fisher markets, Poisson tomography, D-design, and the like. In this setting, the application of existing order-optimal adaptive methods - like UnixGrad or AcceleGrad - is not possible, especially in the presence of randomness and uncertainty. The proposed method - which we call adaptive mirror descent (AdaMir) - aims to close this gap by concurrently achieving min-max optimal rates in problems that are relatively continuous or smooth, including stochastic ones.
Comments: 34 pages, 4 figures
Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG)
MSC classes: Primary 90C25, 90C15, 90C30, secondary 68Q25, 90C60
Cite as: arXiv:2107.08011 [math.OC]
  (or arXiv:2107.08011v1 [math.OC] for this version)

Submission history

From: Panayotis Mertikopoulos [view email]
[v1] Fri, 16 Jul 2021 16:59:40 GMT (1865kb,D)

Link back to: arXiv, form interface, contact.