Current browse context:
math
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Reinforcement Learning with General Value Function Approximation: Provably Efficient Approach via Bounded Eluder Dimension
(Submitted on 21 May 2020 (v1), last revised 19 Jun 2020 (this version, v3))
Abstract: Value function approximation has demonstrated phenomenal empirical success in reinforcement learning (RL). Nevertheless, despite a handful of recent progress on developing theory for RL with linear function approximation, the understanding of general function approximation schemes largely remains missing. In this paper, we establish a provably efficient RL algorithm with general value function approximation. We show that if the value functions admit an approximation with a function class $\mathcal{F}$, our algorithm achieves a regret bound of $\widetilde{O}(\mathrm{poly}(dH)\sqrt{T})$ where $d$ is a complexity measure of $\mathcal{F}$ that depends on the eluder dimension [Russo and Van Roy, 2013] and log-covering numbers, $H$ is the planning horizon, and $T$ is the number interactions with the environment. Our theory generalizes recent progress on RL with linear value function approximation and does not make explicit assumptions on the model of the environment. Moreover, our algorithm is model-free and provides a framework to justify the effectiveness of algorithms used in practice.
Submission history
From: Ruosong Wang [view email][v1] Thu, 21 May 2020 17:36:09 GMT (34kb,D)
[v2] Wed, 17 Jun 2020 18:59:13 GMT (29kb)
[v3] Fri, 19 Jun 2020 17:49:05 GMT (29kb)
Link back to: arXiv, form interface, contact.