Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Sublinear Regret for Learning POMDPs
(Submitted on 8 Jul 2021 (v1), last revised 16 Jul 2022 (this version, v4))
Abstract: We study the model-based undiscounted reinforcement learning for partially observable Markov decision processes (POMDPs). The oracle we consider is the optimal policy of the POMDP with a known environment in terms of the average reward over an infinite horizon. We propose a learning algorithm for this problem, building on spectral method-of-moments estimations for hidden Markov models, the belief error control in POMDPs and upper-confidence-bound methods for online learning. We establish a regret bound of $O(T^{2/3}\sqrt{\log T})$ for the proposed learning algorithm where $T$ is the learning horizon. This is, to the best of our knowledge, the first algorithm achieving sublinear regret with respect to our oracle for learning general POMDPs.
Submission history
From: Xuefeng Gao [view email][v1] Thu, 8 Jul 2021 06:59:39 GMT (130kb,D)
[v2] Wed, 14 Jul 2021 02:41:00 GMT (130kb,D)
[v3] Mon, 27 Sep 2021 05:37:58 GMT (131kb,D)
[v4] Sat, 16 Jul 2022 06:43:01 GMT (136kb,D)
Link back to: arXiv, form interface, contact.