Current browse context:
math.ST
Change to browse by:
References & Citations
Mathematics > Statistics Theory
Title: A Hoeffding Inequality for Finite State Markov Chains and its Applications to Markovian Bandits
(Submitted on 5 Jan 2020 (v1), last revised 10 Jul 2020 (this version, v2))
Abstract: This paper develops a Hoeffding inequality for the partial sums $\sum_{k=1}^n f (X_k)$, where $\{X_k\}_{k \in \mathbb{Z}_{> 0}}$ is an irreducible Markov chain on a finite state space $S$, and $f : S \to [a, b]$ is a real-valued function. Our bound is simple, general, since it only assumes irreducibility and finiteness of the state space, and powerful. In order to demonstrate its usefulness we provide two applications in multi-armed bandit problems. The first is about identifying an approximately best Markovian arm, while the second is concerned with regret minimization in the context of Markovian bandits.
Submission history
From: Vrettos Moulos [view email][v1] Sun, 5 Jan 2020 09:28:10 GMT (12kb,D)
[v2] Fri, 10 Jul 2020 16:56:28 GMT (21kb,D)
Link back to: arXiv, form interface, contact.