Current browse context:
cs.AI
Change to browse by:
References & Citations
Computer Science > Artificial Intelligence
Title: On- and Off-Policy Monotonic Policy Improvement
(Submitted on 10 Oct 2017 (v1), last revised 1 Nov 2017 (this version, v2))
Abstract: Monotonic policy improvement and off-policy learning are two main desirable properties for reinforcement learning algorithms. In this paper, by lower bounding the performance difference of two policies, we show that the monotonic policy improvement is guaranteed from on- and off-policy mixture samples. An optimization procedure which applies the proposed bound can be regarded as an off-policy natural policy gradient method. In order to support the theoretical result, we provide a trust region policy optimization method using experience replay as a naive application of our bound, and evaluate its performance in two classical benchmark problems.
Submission history
From: Ryo Iwaki [view email][v1] Tue, 10 Oct 2017 08:18:24 GMT (14kb)
[v2] Wed, 1 Nov 2017 08:37:34 GMT (157kb,D)
Link back to: arXiv, form interface, contact.