Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Vector Quantized Models for Planning
(Submitted on 8 Jun 2021 (v1), last revised 10 Jun 2021 (this version, v2))
Abstract: Recent developments in the field of model-based RL have proven successful in a range of environments, especially ones where planning is essential. However, such successes have been limited to deterministic fully-observed environments. We present a new approach that handles stochastic and partially-observable environments. Our key insight is to use discrete autoencoders to capture the multiple possible effects of an action in a stochastic environment. We use a stochastic variant of Monte Carlo tree search to plan over both the agent's actions and the discrete latent variables representing the environment's response. Our approach significantly outperforms an offline version of MuZero on a stochastic interpretation of chess where the opponent is considered part of the environment. We also show that our approach scales to DeepMind Lab, a first-person 3D environment with large visual observations and partial observability.
Submission history
From: Sherjil Ozair [view email][v1] Tue, 8 Jun 2021 18:12:32 GMT (13053kb,D)
[v2] Thu, 10 Jun 2021 07:02:41 GMT (13044kb,D)
Link back to: arXiv, form interface, contact.