Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Learn to Exceed: Stereo Inverse Reinforcement Learning with Concurrent Policy Optimization
(Submitted on 21 Sep 2020 (v1), last revised 22 Sep 2020 (this version, v2))
Abstract: In this paper, we study the problem of obtaining a control policy that can mimic and then outperform expert demonstrations in Markov decision processes where the reward function is unknown to the learning agent. One main relevant approach is the inverse reinforcement learning (IRL), which mainly focuses on inferring a reward function from expert demonstrations. The obtained control policy by IRL and the associated algorithms, however, can hardly outperform expert demonstrations. To overcome this limitation, we propose a novel method that enables the learning agent to outperform the demonstrator via a new concurrent reward and action policy learning approach. In particular, we first propose a new stereo utility definition that aims to address the bias in the interpretation of expert demonstrations. We then propose a loss function for the learning agent to learn reward and action policies concurrently such that the learning agent can outperform expert demonstrations. The performance of the proposed method is first demonstrated in OpenAI environments. Further efforts are conducted to experimentally validate the proposed method via an indoor drone flight scenario.
Submission history
From: Yongcan Cao [view email][v1] Mon, 21 Sep 2020 02:16:21 GMT (502kb,D)
[v2] Tue, 22 Sep 2020 23:04:09 GMT (503kb,D)
Link back to: arXiv, form interface, contact.