Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: When is Offline Two-Player Zero-Sum Markov Game Solvable?
(Submitted on 10 Jan 2022 (v1), last revised 14 Oct 2022 (this version, v2))
Abstract: We study what dataset assumption permits solving offline two-player zero-sum Markov games. In stark contrast to the offline single-agent Markov decision process, we show that the single strategy concentration assumption is insufficient for learning the Nash equilibrium (NE) strategy in offline two-player zero-sum Markov games. On the other hand, we propose a new assumption named unilateral concentration and design a pessimism-type algorithm that is provably efficient under this assumption. In addition, we show that the unilateral concentration assumption is necessary for learning an NE strategy. Furthermore, our algorithm can achieve minimax sample complexity without any modification for two widely studied settings: dataset with uniform concentration assumption and turn-based Markov games. Our work serves as an important initial step towards understanding offline multi-agent reinforcement learning.
Submission history
From: Qiwen Cui [view email][v1] Mon, 10 Jan 2022 18:34:32 GMT (47kb)
[v2] Fri, 14 Oct 2022 05:05:34 GMT (29kb)
Link back to: arXiv, form interface, contact.