Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Regret Bounds and Reinforcement Learning Exploration of EXP-based Algorithms
(Submitted on 20 Sep 2020 (v1), last revised 31 May 2022 (this version, v2))
Abstract: EXP-based algorithms are often used for exploration in non-stochastic bandit problems assuming rewards are bounded. We propose a new algorithm, namely EXP4.P, by modifying EXP4 and establish its upper bound of regret in both bounded and unbounded sub-Gaussian contextual bandit settings. The unbounded reward result also holds for a revised version of EXP3.P. Moreover, we provide a lower bound on regret that suggests no sublinear regret can be achieved given short time horizon. All the analyses do not require bounded rewards compared to classical ones. We also extend EXP4.P from contextual bandit to reinforcement learning to incentivize exploration by multiple agents given black-box rewards. The resulting algorithm has been tested on hard-to-explore games and it shows an improvement on exploration compared to state-of-the-art.
Submission history
From: Mengfan Xu [view email][v1] Sun, 20 Sep 2020 22:31:37 GMT (1259kb,D)
[v2] Tue, 31 May 2022 04:24:49 GMT (1307kb,D)
Link back to: arXiv, form interface, contact.