Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Provably Efficient Reinforcement Learning with General Value Function Approximation
(Submitted on 21 May 2020 (this version), latest version 19 Jun 2020 (v3))
Abstract: Value function approximation has demonstrated phenomenal empirical success in reinforcement learning (RL). Nevertheless, despite a handful of recent progress on developing theory for RL with linear function approximation, the understanding of general function approximation schemes largely remains missing. In this paper, we establish the first provable efficiently RL algorithm with general value function approximation. In particular, we show that if the value functions admit an approximation with a function class $\mathcal{F}$, our algorithm achieves a regret bound of $\widetilde{O}(\mathrm{poly}(dH)\sqrt{T})$ where $d$ is a complexity measure of $\mathcal{F}$, $H$ is the planning horizon, and $T$ is the number interactions with the environment. Our theory strictly generalizes recent progress on RL with linear function approximation and does not make explicit assumptions on the model of the environment. Moreover, our algorithm is model-free and provides a framework to justify algorithms used in practice.
Submission history
From: Ruosong Wang [view email][v1] Thu, 21 May 2020 17:36:09 GMT (34kb,D)
[v2] Wed, 17 Jun 2020 18:59:13 GMT (29kb)
[v3] Fri, 19 Jun 2020 17:49:05 GMT (29kb)
Link back to: arXiv, form interface, contact.