Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Exploration--Exploitation in MDPs with Options
(Submitted on 25 Mar 2017 (v1), last revised 17 Apr 2017 (this version, v2))
Abstract: While a large body of empirical results show that temporally-extended actions and options may significantly affect the learning performance of an agent, the theoretical understanding of how and when options can be beneficial in online reinforcement learning is relatively limited. In this paper, we derive an upper and lower bound on the regret of a variant of UCRL using options. While we first analyze the algorithm in the general case of semi-Markov decision processes (SMDPs), we show how these results can be translated to the specific case of MDPs with options and we illustrate simple scenarios in which the regret of learning with options can be \textit{provably} much smaller than the regret suffered when learning with primitive actions.
Submission history
From: Ronan Fruit [view email][v1] Sat, 25 Mar 2017 09:30:31 GMT (471kb)
[v2] Mon, 17 Apr 2017 09:55:01 GMT (471kb)
Link back to: arXiv, form interface, contact.