References & Citations
Computer Science > Logic in Computer Science
Title: Robust Synchronization in Markov Decision Processes
(Submitted on 12 Feb 2014 (v1), last revised 30 Jun 2014 (this version, v3))
Abstract: We consider synchronizing properties of Markov decision processes (MDP), viewed as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in some state, and a sequence of probability distributions is weakly p-synchronizing, or strongly p-synchronizing if respectively infinitely many, or all but finitely many distributions in the sequence are p-synchronizing.
For each synchronizing mode, an MDP can be (i) sure winning if there is a strategy that produces a 1-synchronizing sequence; (ii) almost-sure winning if there is a strategy that produces a sequence that is, for all {\epsilon} > 0, a (1-{\epsilon})-synchronizing sequence; (iii) limit-sure winning if for all {\epsilon} > 0, there is a strategy that produces a (1-{\epsilon})-synchronizing sequence.
For each synchronizing and winning mode, we consider the problem of deciding whether an MDP is winning, and we establish matching upper and lower complexity bounds of the problems, as well as the optimal memory requirement for winning strategies: (a) for all winning modes, we show that the problems are PSPACE-complete for weakly synchronizing, and PTIME-complete for strongly synchronizing; (b) we show that for weakly synchronizing, exponential memory is sufficient and may be necessary for sure winning, and infinite memory is necessary for almost-sure winning; for strongly synchronizing, linear-size memory is sufficient and may be necessary in all modes; (c) we show a robustness result that the almost-sure and limit-sure winning modes coincide for both weakly and strongly synchronizing.
Submission history
From: Mahsa Shirmohammadi [view email][v1] Wed, 12 Feb 2014 14:47:05 GMT (77kb)
[v2] Fri, 18 Apr 2014 17:00:26 GMT (79kb)
[v3] Mon, 30 Jun 2014 15:42:47 GMT (80kb)
Link back to: arXiv, form interface, contact.