cs.PF

# Title: Ergodic Theory for Controlled Markov Chains with Stationary Inputs

Abstract: Consider a stochastic process $\{X(t)\}$ on a finite state space ${\sf X}=\{1,\dots, d\}$. It is conditionally Markov, given a real-valued `input process' $\{\zeta(t)\}$. This is assumed to be small, which is modeled through the scaling, $\zeta_t = \varepsilon \zeta^1_t, \qquad 0\le \varepsilon \le 1\,,$ where $\{\zeta^1(t)\}$ is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on $\{\zeta(t)\}$:
(i) A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain $\{X^\bullet$(t)\}obtained with $\{\zeta(t)\}\equiv 0$. The triple $(\{X(t)\}, \{X^\bullet(t)\},\{\zeta(t)\})$ is a jointly stationary process satisfying ${\sf P}\{X(t) \neq X^\bullet(t)\} = O(\varepsilon)$ Moreover, a second-order Taylor-series approximation is obtained: ${\sf P}\{X(t) =i \} ={\sf P}\{X^\bullet(t) =i \} + \varepsilon^2 \varrho(i) + o(\varepsilon^2),\quad 1\le i\le d,$ with an explicit formula for the vector $\varrho\in\mathbb{R}^d$.
(ii) For any $m\ge 1$ and any function $f\colon \{1,\dots,d\}\times \mathbb{R}\to\mathbb{R}^m$, the stationary stochastic process $Y(t) = f(X(t),\zeta(t))$ has a power spectral density $\text{S}_f$ that admits a second order Taylor series expansion: A function $\text{S}^{(2)}_f\colon [-\pi,\pi] \to \mathbb{C}^{ m\times m}$ is constructed such that $\text{S}_f(\theta) = \text{S}^\bullet_f(\theta) + \varepsilon^2 \text{S}_f^{(2)}(\theta) + o(\varepsilon^2),\quad \theta\in [-\pi,\pi] .$ An explicit formula for the function $\text{S}_f^{(2)}$ is obtained, based in part on the bounds in (i).
The results are illustrated using a version of the timing channel of Anantharam and Verdu.
 Subjects: Performance (cs.PF); Information Theory (cs.IT); Applications (stat.AP) MSC classes: 60J20, 60G10, 68M20, 94A15 DOI: 10.1214/17-AAP1300 Cite as: arXiv:1604.04013 [cs.PF] (or arXiv:1604.04013v2 [cs.PF] for this version)

## Submission history

From: Sean Meyn [view email]
[v1] Thu, 14 Apr 2016 02:16:52 GMT (1343kb,D)
[v2] Sat, 18 Jun 2016 09:39:26 GMT (1343kb,D)

Link back to: arXiv, form interface, contact.