Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Smaller World Models for Reinforcement Learning
(Submitted on 12 Oct 2020 (v1), last revised 2 Mar 2021 (this version, v2))
Abstract: Sample efficiency remains a fundamental issue of reinforcement learning. Model-based algorithms try to make better use of data by simulating the environment with a model. We propose a new neural network architecture for world models based on a vector quantized-variational autoencoder (VQ-VAE) to encode observations and a convolutional LSTM to predict the next embedding indices. A model-free PPO agent is trained purely on simulated experience from the world model. We adopt the setup introduced by Kaiser et al. (2020), which only allows 100K interactions with the real environment. We apply our method on 36 Atari environments and show that we reach comparable performance to their SimPLe algorithm, while our model is significantly smaller.
Submission history
From: Jan Robine [view email][v1] Mon, 12 Oct 2020 15:02:41 GMT (231kb,D)
[v2] Tue, 2 Mar 2021 12:02:16 GMT (224kb,D)
Link back to: arXiv, form interface, contact.