Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Safe Policy Improvement with Baseline Bootstrapping
(Submitted on 19 Dec 2017 (this version), latest version 7 Jun 2019 (v5))
Abstract: A common goal in Reinforcement Learning is to derive a good strategy given a limited batch of data. In this paper, we adopt the safe policy improvement (SPI) approach: we compute a target policy guaranteed to perform at least as well as a given baseline policy. Our SPI strategy, inspired by the knows-what-it-knows paradigms, consists in bootstrapping the target policy with the baseline policy when it does not know. We develop two computationally efficient bootstrapping algorithms, a value-based and a policy-based, both accompanied with theoretical SPI bounds. Three algorithm variants are proposed. We empirically show the literature algorithms limits on a small stochastic gridworld problem, and then demonstrate that our five algorithms not only improve the worst case scenarios, but also the mean performance.
Submission history
From: Romain Laroche [view email][v1] Tue, 19 Dec 2017 13:43:41 GMT (11866kb,D)
[v2] Wed, 20 Dec 2017 19:52:03 GMT (11866kb,D)
[v3] Thu, 18 Jan 2018 21:37:53 GMT (8422kb,D)
[v4] Thu, 14 Jun 2018 19:54:34 GMT (8002kb,D)
[v5] Fri, 7 Jun 2019 17:45:54 GMT (18526kb,D)
Link back to: arXiv, form interface, contact.