Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces
(Submitted on 9 Jul 2020 (v1), last revised 23 Mar 2022 (this version, v2))
Abstract: In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric. Using a non-parametric model of the MDP built with time-dependent kernels, we prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time, which quantifies its level of non-stationarity. Our method generalizes previous approaches based on sliding windows and exponential discounting used to handle changing environments. We further propose a practical implementation of KeRNS, we analyze its regret and validate it experimentally.
Submission history
From: Omar Darwiche Domingues [view email][v1] Thu, 9 Jul 2020 21:37:13 GMT (1121kb,D)
[v2] Wed, 23 Mar 2022 20:21:47 GMT (12366kb,D)
Link back to: arXiv, form interface, contact.