Current browse context:
math.ST
Change to browse by:
References & Citations
Mathematics > Statistics Theory
Title: Batch Policy Learning in Average Reward Markov Decision Processes
(Submitted on 23 Jul 2020 (v1), last revised 17 Sep 2022 (this version, v3))
Abstract: We consider the batch (off-line) policy learning problem in the infinite horizon Markov Decision Process. Motivated by mobile health applications, we focus on learning a policy that maximizes the long-term average reward. We propose a doubly robust estimator for the average reward and show that it achieves semiparametric efficiency. Further we develop an optimization algorithm to compute the optimal policy in a parameterized stochastic policy class. The performance of the estimated policy is measured by the difference between the optimal average reward in the policy class and the average reward of the estimated policy and we establish a finite-sample regret guarantee. The performance of the method is illustrated by simulation studies and an analysis of a mobile health study promoting physical activity.
Submission history
From: Peng Liao [view email][v1] Thu, 23 Jul 2020 03:28:14 GMT (123kb)
[v2] Tue, 9 Nov 2021 05:36:30 GMT (187kb,D)
[v3] Sat, 17 Sep 2022 17:56:24 GMT (173kb,D)
Link back to: arXiv, form interface, contact.