Current browse context:
math.ST
Change to browse by:
References & Citations
Mathematics > Statistics Theory
Title: Asymptotic Optimality in Stochastic Optimization
(Submitted on 16 Dec 2016 (v1), last revised 2 Nov 2018 (this version, v4))
Abstract: We study local complexity measures for stochastic convex optimization problems, providing a local minimax theory analogous to that of H\'{a}jek and Le Cam for classical statistical problems. We give complementary optimality results, developing fully online methods that adaptively achieve optimal convergence guarantees. Our results provide function-specific lower bounds and convergence results that make precise a correspondence between statistical difficulty and the geometric notion of tilt-stability from optimization. As part of this development, we show how variants of Nesterov's dual averaging---a stochastic gradient-based procedure---guarantee finite time identification of constraints in optimization problems, while stochastic gradient procedures fail. Additionally, we highlight a gap between problems with linear and nonlinear constraints: standard stochastic-gradient-based procedures are suboptimal even for the simplest nonlinear constraints, necessitating the development of asymptotically optimal Riemannian stochastic gradient methods.
Submission history
From: Feng Ruan [view email][v1] Fri, 16 Dec 2016 19:54:22 GMT (69kb,D)
[v2] Wed, 2 Aug 2017 09:08:42 GMT (80kb,D)
[v3] Thu, 3 Aug 2017 02:32:45 GMT (80kb,D)
[v4] Fri, 2 Nov 2018 09:10:18 GMT (1003kb,D)
Link back to: arXiv, form interface, contact.