Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Learning Adversarial MDPs with Bandit Feedback and Unknown Transition
(Submitted on 3 Dec 2019 (v1), last revised 2 Nov 2020 (this version, v5))
Abstract: We consider the problem of learning in episodic finite-horizon Markov decision processes with an unknown transition function, bandit feedback, and adversarial losses. We propose an efficient algorithm that achieves $\mathcal{\tilde{O}}(L|X|\sqrt{|A|T})$ regret with high probability, where $L$ is the horizon, $|X|$ is the number of states, $|A|$ is the number of actions, and $T$ is the number of episodes. To the best of our knowledge, our algorithm is the first to ensure $\mathcal{\tilde{O}}(\sqrt{T})$ regret in this challenging setting; in fact it achieves the same regret bound as (Rosenberg & Mansour, 2019a) that considers an easier setting with full-information feedback. Our key technical contributions are two-fold: a tighter confidence set for the transition function, and an optimistic loss estimator that is inversely weighted by an $\textit{upper occupancy bound}$.
Submission history
From: Tiancheng Jin [view email][v1] Tue, 3 Dec 2019 05:04:40 GMT (273kb)
[v2] Thu, 5 Dec 2019 05:06:43 GMT (278kb)
[v3] Tue, 7 Jan 2020 04:36:59 GMT (397kb)
[v4] Tue, 13 Oct 2020 13:38:22 GMT (66kb)
[v5] Mon, 2 Nov 2020 07:13:30 GMT (401kb)
Link back to: arXiv, form interface, contact.