### Current browse context:

math.PR

### Change to browse by:

### References & Citations

# Mathematics > Probability

# Title: Probabilistic Contraction Analysis of Iterated Random Operators

(Submitted on 4 Apr 2018 (v1), last revised 15 Jul 2020 (this version, v5))

Abstract: Consider a contraction operator $T$ over a complete metric space $\mathcal X$ with the fixed point $x^\star$. In many computational applications, it is difficult to compute $T(x)$; therefore, one replaces the application contraction operator $T$ at iteration $k$ by a random operator $\hat T^n_k$ using $n$ independent and identically distributed samples of a random variable. Consider the Markov chain $(\hat X^n_k)_{k\in\mathbb{N}}$, which is generated by $\hat X^n_{k+1} = \hat T^n_k(\hat X^n_k)$. In this paper, we identify some sufficient conditions under which (i) the distribution of $\hat X^n_k$ converges to a Dirac mass over $x^\star$ as $k$ and $n$ go to infinity, and (ii) the probability that $\hat X^n_k$ is far from $x^\star$ as $k$ goes to infinity can be made arbitrarily small by an appropriate choice of $n$. We also derive an upper bound on the probability that $\hat X^n_k$ is far from $x^\star$ as $k\rightarrow \infty$. We apply the result to study the convergence in probability of iterates generated by empirical value iteration algorithms for discounted and average cost Markov decision problems.

## Submission history

From: Abhishek Gupta [view email]**[v1]**Wed, 4 Apr 2018 00:10:58 GMT (25kb)

**[v2]**Mon, 23 Apr 2018 18:40:47 GMT (25kb)

**[v3]**Sun, 10 Feb 2019 00:49:26 GMT (28kb)

**[v4]**Tue, 12 Feb 2019 11:05:06 GMT (28kb)

**[v5]**Wed, 15 Jul 2020 18:18:12 GMT (78kb)

Link back to: arXiv, form interface, contact.