We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

math.OC

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Mathematics > Optimization and Control

Title: A Non-asymptotic Analysis of Non-parametric Temporal-Difference Learning

Authors: Eloïse Berthier (SIERRA, PSL), Ziad Kobeissi (SIERRA, PSL), Francis Bach (SIERRA, PSL)
Abstract: Temporal-difference learning is a popular algorithm for policy evaluation. In this paper, we study the convergence of the regularized non-parametric TD(0) algorithm, in both the independent and Markovian observation settings. In particular, when TD is performed in a universal reproducing kernel Hilbert space (RKHS), we prove convergence of the averaged iterates to the optimal value function, even when it does not belong to the RKHS. We provide explicit convergence rates that depend on a source condition relating the regularity of the optimal value function to the RKHS. We illustrate this convergence numerically on a simple continuous-state Markov reward process.
Subjects: Optimization and Control (math.OC)
Cite as: arXiv:2205.11831 [math.OC]
  (or arXiv:2205.11831v1 [math.OC] for this version)

Submission history

From: Eloise Berthier [view email]
[v1] Tue, 24 May 2022 06:52:44 GMT (2774kb,D)

Link back to: arXiv, form interface, contact.