References & Citations
Computer Science > Machine Learning
Title: Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approach
(Submitted on 31 Jan 2022 (v1), last revised 11 Oct 2022 (this version, v3))
Abstract: We present BRIEE (Block-structured Representation learning with Interleaved Explore Exploit), an algorithm for efficient reinforcement learning in Markov Decision Processes with block-structured dynamics (i.e., Block MDPs), where rich observations are generated from a set of unknown latent states. BRIEE interleaves latent states discovery, exploration, and exploitation together, and can provably learn a near-optimal policy with sample complexity scaling polynomially in the number of latent states, actions, and the time horizon, with no dependence on the size of the potentially infinite observation space. Empirically, we show that BRIEE is more sample efficient than the state-of-art Block MDP algorithm HOMER and other empirical RL baselines on challenging rich-observation combination lock problems that require deep exploration.
Submission history
From: Xuezhou Zhang [view email][v1] Mon, 31 Jan 2022 19:47:55 GMT (909kb,D)
[v2] Wed, 2 Feb 2022 19:21:15 GMT (909kb,D)
[v3] Tue, 11 Oct 2022 04:12:57 GMT (981kb,D)
Link back to: arXiv, form interface, contact.