We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Mathematics > Optimization and Control

Title: The Last-Iterate Convergence Rate of Optimistic Mirror Descent in Stochastic Variational Inequalities

Abstract: In this paper, we analyze the local convergence rate of optimistic mirror descent methods in stochastic variational inequalities, a class of optimization problems with important applications to learning theory and machine learning. Our analysis reveals an intricate relation between the algorithm's rate of convergence and the local geometry induced by the method's underlying Bregman function. We quantify this relation by means of the Legendre exponent, a notion that we introduce to measure the growth rate of the Bregman divergence relative to the ambient norm near a solution. We show that this exponent determines both the optimal step-size policy of the algorithm and the optimal rates attained, explaining in this way the differences observed for some popular Bregman functions (Euclidean projection, negative entropy, fractional power, etc.).
Comments: 31 pages, 3 figures, 1 table; to be presented at the 34th Annual Conference on Learning Theory (COLT 2021)
Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG)
MSC classes: 65K15, 90C33 (Primary) 68Q25, 68Q32 (Secondary)
Cite as: arXiv:2107.01906 [math.OC]
  (or arXiv:2107.01906v1 [math.OC] for this version)

Submission history

From: Waïss Azizian [view email]
[v1] Mon, 5 Jul 2021 09:54:47 GMT (109kb)

Link back to: arXiv, form interface, contact.