Title: Ergodic Theory for Controlled Markov Chains with Stationary Inputs

Abstract: Consider a stochastic process $\Gamma$ on the finite alphabet consisting of the $d$ standard basis elements in $\Re^d$. It is conditionally Markov, given a real-valued `input process' $\zeta$. This is assumed to be small, which is modeled through the scaling, \[ \zeta_t = \varepsilon \zeta^1_t, \qquad 0\le \varepsilon \le 1\,, \] where $\zeta^1$ is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on $\zeta$:
(i) A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain $\Gamma^\bullet$ obtained with $\zeta\equiv 0$. The triple $(\Gamma, \Gamma^\bullet,\zeta)$ is a jointly stationary process on $\Re^{2d+1}$, satisfying \[ {\sf P}\{\Gamma(t) \neq \Gamma^\bullet(t)\} = O(\varepsilon) \]
(ii) The power spectral density $\text{S}$ for the pair process $(\Gamma, \zeta)$ admits a second order Taylor series expansion: A function $\text{S}^{(2)}\colon [-\pi,\pi] \to \mathbb{C}^{(d+1)\times(d+1)}$ is constructed such that \[ \text{S}(\theta) = \text{S}^\bullet(\theta) + \varepsilon^2\text{S}^{(2)}(\theta) + o(\varepsilon^2),\quad \theta\in [-\pi,\pi] . \]
An explicit formula for the function $\text{S}^{(2)}$ is obtained, based on the coupling bound (i), and extensions of perturbation theory for Markov chains.
The results are illustrated using a version of the timing channel of Anantharam and Verdu.