Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control
(Submitted on 5 Nov 2018 (v1), last revised 28 Jan 2019 (this version, v3))
Abstract: We propose a plan online and learn offline (POLO) framework for the setting where an agent, with an internal model, needs to continually act and learn in the world. Our work builds on the synergistic relationship between local model-based control, global value function learning, and exploration. We study how local trajectory optimization can cope with approximation errors in the value function, and can stabilize and accelerate value function learning. Conversely, we also study how approximate value functions can help reduce the planning horizon and allow for better policies beyond local solutions. Finally, we also demonstrate how trajectory optimization can be used to perform temporally coordinated exploration in conjunction with estimating uncertainty in value function approximation. This exploration is critical for fast and stable learning of the value function. Combining these components enable solutions to complex simulated control tasks, like humanoid locomotion and dexterous in-hand manipulation, in the equivalent of a few minutes of experience in the real world.
Submission history
From: Aravind Rajeswaran [view email][v1] Mon, 5 Nov 2018 17:09:18 GMT (3648kb,D)
[v2] Thu, 27 Dec 2018 20:54:47 GMT (3649kb,D)
[v3] Mon, 28 Jan 2019 16:39:23 GMT (3649kb,D)
Link back to: arXiv, form interface, contact.