### Current browse context:

cs.PF

### Change to browse by:

### References & Citations

# Computer Science > Performance

# Title: Ergodic Theory for Controlled Markov Chains with Stationary Inputs

(Submitted on 14 Apr 2016 (v1), last revised 18 Jun 2016 (this version, v2))

Abstract: Consider a stochastic process $\{X(t)\}$ on a finite state space $ {\sf X}=\{1,\dots, d\}$. It is conditionally Markov, given a real-valued `input process' $\{\zeta(t)\}$. This is assumed to be small, which is modeled through the scaling, \[ \zeta_t = \varepsilon \zeta^1_t, \qquad 0\le \varepsilon \le 1\,, \] where $\{\zeta^1(t)\}$ is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on $\{\zeta(t)\}$:

(i) A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain $\{X^\bullet$(t)\}obtained with $\{\zeta(t)\}\equiv 0$. The triple $(\{X(t)\}, \{X^\bullet(t)\},\{\zeta(t)\})$ is a jointly stationary process satisfying \[ {\sf P}\{X(t) \neq X^\bullet(t)\} = O(\varepsilon) \] Moreover, a second-order Taylor-series approximation is obtained: \[ {\sf P}\{X(t) =i \} ={\sf P}\{X^\bullet(t) =i \} + \varepsilon^2 \varrho(i) + o(\varepsilon^2),\quad 1\le i\le d, \] with an explicit formula for the vector $\varrho\in\mathbb{R}^d$.

(ii) For any $m\ge 1$ and any function $f\colon \{1,\dots,d\}\times \mathbb{R}\to\mathbb{R}^m$, the stationary stochastic process $Y(t) = f(X(t),\zeta(t))$ has a power spectral density $\text{S}_f$ that admits a second order Taylor series expansion: A function $\text{S}^{(2)}_f\colon [-\pi,\pi] \to \mathbb{C}^{ m\times m}$ is constructed such that \[ \text{S}_f(\theta) = \text{S}^\bullet_f(\theta) + \varepsilon^2 \text{S}_f^{(2)}(\theta) + o(\varepsilon^2),\quad \theta\in [-\pi,\pi] . \] An explicit formula for the function $\text{S}_f^{(2)}$ is obtained, based in part on the bounds in (i).

The results are illustrated using a version of the timing channel of Anantharam and Verdu.

## Submission history

From: Sean Meyn [view email]**[v1]**Thu, 14 Apr 2016 02:16:52 GMT (1343kb,D)

**[v2]**Sat, 18 Jun 2016 09:39:26 GMT (1343kb,D)

Link back to: arXiv, form interface, contact.