Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Exploration via Hindsight Goal Generation
(Submitted on 10 Jun 2019 (v1), last revised 18 Dec 2019 (this version, v3))
Abstract: Goal-oriented reinforcement learning has recently been a practical framework for robotic manipulation tasks, in which an agent is required to reach a certain goal defined by a function on the state space. However, the sparsity of such reward definition makes traditional reinforcement learning algorithms very inefficient. Hindsight Experience Replay (HER), a recent advance, has greatly improved sample efficiency and practical applicability for such problems. It exploits previous replays by constructing imaginary goals in a simple heuristic way, acting like an implicit curriculum to alleviate the challenge of sparse reward signal. In this paper, we introduce Hindsight Goal Generation (HGG), a novel algorithmic framework that generates valuable hindsight goals which are easy for an agent to achieve in the short term and are also potential for guiding the agent to reach the actual goal in the long term. We have extensively evaluated our goal generation algorithm on a number of robotic manipulation tasks and demonstrated substantially improvement over the original HER in terms of sample efficiency.
Submission history
From: Zhizhou Ren [view email][v1] Mon, 10 Jun 2019 21:21:18 GMT (5861kb,D)
[v2] Thu, 5 Dec 2019 05:35:33 GMT (3407kb,D)
[v3] Wed, 18 Dec 2019 04:31:39 GMT (3250kb,D)
Link back to: arXiv, form interface, contact.