We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Probability

Title: Probabilistic Contraction Analysis of Iterated Random Operators

Abstract: Consider a contraction operator $T$ over a complete metric space $\mathcal X$ with the fixed point $x^\star$. In many computational applications, it is difficult to compute $T(x)$; therefore, one replaces the application contraction operator $T$ at iteration $k$ by a random operator $\hat T^n_k$ using $n$ independent and identically distributed samples of a random variable. Consider the Markov chain $(\hat X^n_k)_{k\in\mathbb{N}}$, which is generated by $\hat X^n_{k+1} = \hat T^n_k(\hat X^n_k)$. In this paper, we identify some sufficient conditions under which (i) the distribution of $\hat X^n_k$ converges to a Dirac mass over $x^\star$ as $k$ and $n$ go to infinity, and (ii) the probability that $\hat X^n_k$ is far from $x^\star$ as $k$ goes to infinity can be made arbitrarily small by an appropriate choice of $n$. We also derive an upper bound on the probability that $\hat X^n_k$ is far from $x^\star$ as $k\rightarrow \infty$. We apply the result to study the convergence in probability of iterates generated by empirical value iteration algorithms for discounted and average cost Markov decision problems.
Comments: 37 pages, submitted to SIAM Journal on Control and Optimization
Subjects: Probability (math.PR); Machine Learning (cs.LG); Optimization and Control (math.OC)
Cite as: arXiv:1804.01195 [math.PR]
  (or arXiv:1804.01195v5 [math.PR] for this version)

Submission history

From: Abhishek Gupta [view email]
[v1] Wed, 4 Apr 2018 00:10:58 GMT (25kb)
[v2] Mon, 23 Apr 2018 18:40:47 GMT (25kb)
[v3] Sun, 10 Feb 2019 00:49:26 GMT (28kb)
[v4] Tue, 12 Feb 2019 11:05:06 GMT (28kb)
[v5] Wed, 15 Jul 2020 18:18:12 GMT (78kb)

Link back to: arXiv, form interface, contact.