We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: Nonlinear Two-Time-Scale Stochastic Approximation: Convergence and Finite-Time Performance

Authors: Thinh T. Doan
Abstract: Two-time-scale stochastic approximation, a generalized version of the popular stochastic approximation, has found broad applications in many areas including stochastic control, optimization, and machine learning. Despite its popularity, theoretical guarantees of this method, especially its finite-time performance, are mostly achieved for the linear case while the results for the nonlinear counterpart are very sparse. Motivated by the classic control theory for singularly perturbed systems, we study in this paper the asymptotic convergence and finite-time analysis of the nonlinear two-time-scale stochastic approximation. Under some fairly standard assumptions, we provide a formula that characterizes the rate of convergence of the main iterates to the desired solutions. In particular, we show that the method achieves a convergence in expectation at a rate $\mathcal{O}(1/k^{2/3})$, where $k$ is the number of iterations. The key idea in our analysis is to properly choose the two step sizes to characterize the coupling between the fast and slow-time-scale iterates.
Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Systems and Control (eess.SY)
Cite as: arXiv:2011.01868 [math.OC]
  (or arXiv:2011.01868v3 [math.OC] for this version)

Submission history

From: Thinh Doan [view email]
[v1] Tue, 3 Nov 2020 17:43:39 GMT (31kb)
[v2] Tue, 2 Mar 2021 15:26:28 GMT (33kb)
[v3] Tue, 23 Mar 2021 13:44:57 GMT (21kb)

Link back to: arXiv, form interface, contact.